Oct 30 21:33:03.896261 ip-10-0-130-104 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:33:04.258839 ip-10-0-130-104 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:33:04.258839 ip-10-0-130-104 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:33:04.258839 ip-10-0-130-104 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:33:04.258839 ip-10-0-130-104 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:33:04.262771 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262689    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:33:04.262771 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262772    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:33:04.262771 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262776    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262780    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262784    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262786    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262789    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262792    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262794    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262797    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262800    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262803    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262805    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262808    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262811    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262813    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262816    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262818    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262821    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262824    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262828    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262830    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:33:04.262864 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262833    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262838    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262842    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262845    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262848    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262851    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262853    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262856    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262859    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262861    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262864    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262866    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262869    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262872    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262875    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262878    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262881    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262889    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262892    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:33:04.263339 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262894    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262897    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262899    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262901    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262916    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262919    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262922    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262924    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262927    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262929    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262932    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262934    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262938    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262940    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262944    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262947    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262949    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262952    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262955    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:33:04.263773 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262957    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262960    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262962    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262971    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262974    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262977    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262979    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262982    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262984    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262987    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262990    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.262992    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263002    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263006    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263009    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263012    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263015    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263017    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263020    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:33:04.264241 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263022    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263025    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263028    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263030    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263033    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263035    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263037    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263040    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263043    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263046    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263048    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263051    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263054    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263056    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263058    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263061    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263064    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263066    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263069    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263072    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:33:04.264769 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263074    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263077    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263079    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263082    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263084    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263087    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.263099    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264567    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264574    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264577    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264580    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264583    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264586    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264588    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264591    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264594    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264597    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264600    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264603    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264606    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:33:04.265248 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264608    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264612    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264615    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264618    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264620    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264623    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264627    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264631    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264635    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264638    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264641    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264643    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264646    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264649    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264651    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264654    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264658    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264662    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264665    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:33:04.265721 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264668    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264671    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264674    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264676    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264679    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264681    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264684    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264687    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264689    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264692    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264694    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264697    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264700    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264702    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264705    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264708    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264712    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264715    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264717    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264720    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:33:04.266194 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264723    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264725    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264728    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264730    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264733    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264735    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264738    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264740    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264743    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264745    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264748    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264750    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264753    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264756    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264759    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264761    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264764    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264767    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264773    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264776    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:33:04.266664 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264779    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264782    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264785    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264788    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264790    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264793    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264795    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264798    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264801    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264804    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264806    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264809    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264811    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264814    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264816    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264819    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264821    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264824    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264826    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264829    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:33:04.267149 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264831    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264834    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264836    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264839    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264842    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264844    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264848    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264852    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264854    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264857    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264861    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264864    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264867    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.264869    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265352    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265356    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265359    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265362    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265364    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:33:04.267613 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265367    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265370    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265372    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265375    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265378    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265380    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265383    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265386    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265388    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265391    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265394    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265396    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265399    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265402    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265404    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265407    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265409    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265412    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265414    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265418    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:33:04.268083 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265420    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265423    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265426    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265429    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265431    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265433    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265436    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265439    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265441    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265443    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265446    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265448    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265451    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265454    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265457    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265460    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265463    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265466    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265468    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:33:04.268561 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265471    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265474    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265476    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265478    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265481    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265483    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265486    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265488    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265490    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265493    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265496    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265498    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265500    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265503    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265506    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265508    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265511    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265514    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265516    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265519    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:33:04.269037 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265521    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265524    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265526    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265529    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265531    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265534    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265536    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265541    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265544    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265546    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265549    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265552    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265554    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265557    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265559    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265562    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265565    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265567    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265570    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265572    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:33:04.269498 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265575    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265577    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265580    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265584    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265587    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265590    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265593    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265596    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265600    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265604    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265607    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265609    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265612    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265615    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265617    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265619    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265622    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265624    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265627    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:33:04.269973 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265632    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265634    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.265637    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267851    2574 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267861    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267867    2574 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267872    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267877    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267880    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267885    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267889    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267892    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267895    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267899    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267902    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267916    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267920    2574 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267923    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267926    2574 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267929    2574 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267934    2574 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267938    2574 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267941    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267944    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:33:04.270446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267947    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267951    2574 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267954    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267957    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267961    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267964    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267967    2574 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267970    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267974    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267978    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267981    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267985    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267988    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267991    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267994    2574 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.267997    2574 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268000    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268004    2574 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268007    2574 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268010    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268014    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268017    2574 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268021    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268024    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268027    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:33:04.271024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268030    2574 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268033    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268036    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268041    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268045    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268048    2574 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268051    2574 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268054    2574 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268058    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268062    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268065    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268068    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268071    2574 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268074    2574 flags.go:64] FLAG: --help="false"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268077    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268081    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268084    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268087    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268090    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268094    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268097    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268101    2574 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268103    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:33:04.271662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268107    2574 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268109    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268113    2574 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268115    2574 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268119    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268122    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268125    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268127    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268131    2574 flags.go:64] FLAG: --lock-file=""
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268134    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268137    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268140    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268147    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268150    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268154    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268157    2574 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268161    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268164    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268167    2574 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268170    2574 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268175    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268178    2574 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268181    2574 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268184    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268188    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:33:04.272224 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268191    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268194    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268197    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268201    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268204    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268211    2574 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268214    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268217    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268221    2574 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268224    2574 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268227    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268231    2574 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268234    2574 flags.go:64] FLAG: --port="10250"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268237    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268240    2574 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-00fb5a58d9fc2e9b1"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268243    2574 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268246    2574 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268249    2574 flags.go:64] FLAG: --register-node="true"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268252    2574 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268258    2574 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268261    2574 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268264    2574 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268267    2574 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268271    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268275    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268278    2574 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:33:04.272813 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268281    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268284    2574 flags.go:64] FLAG: --runonce="false"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268287    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268297    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268301    2574 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268304    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268306    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268310    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268313    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268316    2574 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268319    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268322    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268325    2574 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268328    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268331    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268334    2574 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268336    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268342    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268345    2574 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268347    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268352    2574 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268355    2574 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268357    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268360    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268363    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:33:04.273432 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268369    2574 flags.go:64] FLAG: --v="2"
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268373    2574 flags.go:64] FLAG: --version="false"
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268378    2574 flags.go:64] FLAG: --vmodule=""
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268382    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268386    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268535    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268539    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268542    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268545    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268547    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268550    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268553    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268555    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268558    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268561    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268564    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268567    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268569    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268572    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268575    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268577    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:33:04.274025 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268580    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268583    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268586    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268588    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268591    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268593    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268596    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268598    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268601    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268603    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268606    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268611    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268613    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268616    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268618    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268621    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268624    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268627    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268629    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:33:04.274552 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268632    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268634    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268637    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268640    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268642    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268645    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268647    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268649    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268652    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268654    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268657    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268659    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268662    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268664    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268667    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268670    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268672    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268674    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268677    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:33:04.275020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268681    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268684    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268688    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268692    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268694    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268699    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268702    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268705    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268707    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268710    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268712    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268716    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268718    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268721    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268724    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268726    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268729    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268731    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268733    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:33:04.275502 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268736    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268738    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268741    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268743    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268746    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268749    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268751    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268754    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268757    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268761    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268764    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268767    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268770    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268772    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268775    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268777    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268780    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268783    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268787    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:33:04.276199 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268790    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268792    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268795    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268797    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268799    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268802    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268805    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268823    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268827    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268830    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268833    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268836    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268838    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268841    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.268869    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268917    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:33:04.276946 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268922    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268925    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268928    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268931    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268935    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268939    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268942    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268945    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268947    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268950    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268953    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268956    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268958    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268961    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268963    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268968    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268970    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268973    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268976    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:33:04.277584 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268978    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268981    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268983    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268986    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268989    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268991    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268994    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268996    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.268999    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269002    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269004    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269007    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269010    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269014    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269017    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269019    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269022    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269024    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269027    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:33:04.278082 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269029    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269032    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269034    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269038    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269041    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269044    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269046    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269049    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269052    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269056    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269058    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269061    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269063    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269066    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269068    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269071    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269073    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269076    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269079    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269082    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:33:04.278527 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269084    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269087    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269089    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269091    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269094    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269097    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269100    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269103    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269106    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269109    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269111    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269114    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269117    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269119    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269122    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269127    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269130    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269133    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269135    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269138    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:33:04.279020 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269140    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269145    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269148    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269151    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269153    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269156    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269158    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269161    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269163    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269166    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269169    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269172    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269175    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269177    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269179    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269183    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269185    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269188    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269191    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:33:04.279491 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269193    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269196    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269198    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269200    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269203    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269205    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269208    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.269210    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.276928    2574 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.276951    2574 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.276976    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.276984    2574 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277124    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277140    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277144    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277151    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277154    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277157    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277160    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:33:04.279969 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277163    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277168    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277173    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277176    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277202    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277205    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277221    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277225    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277355    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277359    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277362    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277365    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277368    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277370    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277373    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277375    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277378    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277381    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277384    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:33:04.280425 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277387    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277390    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277393    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277395    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277398    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277401    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277404    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277408    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277410    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277413    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277416    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277418    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277421    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277424    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277426    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277429    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277432    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277434    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277436    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:33:04.280886 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277439    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277442    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277444    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277447    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277449    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277452    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277455    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277457    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277460    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277462    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277465    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277468    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277470    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277472    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277475    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277478    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277480    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277484    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277487    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:33:04.281495 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277490    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277493    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277496    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277499    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277501    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277504    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277506    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277509    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277511    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277514    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277519    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277522    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277525    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277528    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277531    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277534    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277536    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277539    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277541    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277544    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:33:04.281977 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277546    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277549    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277552    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277554    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277557    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277559    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277562    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277564    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277567    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277570    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277572    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277575    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277578    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277581    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277584    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277587    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277590    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277593    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277598    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277601    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:33:04.282473 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277603    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277608    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.277647    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277793    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277799    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277803    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277806    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277809    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277814    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277818    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277820    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277823    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277826    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277829    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277832    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277835    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:33:04.283129 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277838    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277840    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277843    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277846    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277848    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277851    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277854    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277856    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277859    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277863    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277866    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277868    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277871    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277874    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277877    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277879    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277882    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277885    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277887    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277890    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:33:04.283509 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277892    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277896    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277900    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277921    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277925    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277930    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277934    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277938    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277941    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277944    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277947    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277950    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277953    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277955    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277958    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277961    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277963    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277966    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277969    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:33:04.284024 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277971    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277975    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277978    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277980    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277983    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277985    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277988    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277991    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277993    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277995    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.277998    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278001    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278003    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278006    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278008    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278011    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278014    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278016    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278019    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278022    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:33:04.284675 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278024    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278027    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278029    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278032    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278034    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278037    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278040    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278042    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278045    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278047    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278050    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278052    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278056    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278058    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278061    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278064    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278067    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278069    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278072    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:33:04.285537 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278074    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278077    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278079    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278082    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278085    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278087    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278090    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278093    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278096    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278099    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278101    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278104    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278107    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278110    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278113    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.278137    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:33:04.286075 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278197    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278202    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278205    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278208    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278211    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278214    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278217    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278220    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278223    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278225    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278228    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278231    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278234    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278236    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278240    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278242    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278245    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278248    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278250    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:33:04.286448 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278253    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278256    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278258    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278261    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278264    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278266    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278269    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278272    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278274    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278277    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278281    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278286    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278290    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278292    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278296    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278300    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278303    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278306    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278309    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:33:04.287115 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278311    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278314    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278316    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278319    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278321    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278324    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278327    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278330    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278332    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278335    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278337    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278340    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278343    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278345    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278348    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278350    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278352    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278355    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278357    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278360    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:33:04.287590 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278363    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278365    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278368    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278370    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278373    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278376    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278378    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278381    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278384    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278386    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278388    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278391    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278393    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278396    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278398    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278401    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278404    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278406    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278409    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278411    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:33:04.288071 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278416    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278419    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278422    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278425    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278427    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278429    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278432    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278434    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278437    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278439    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278442    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278444    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278447    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278449    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278452    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278454    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278457    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278459    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278462    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:33:04.288555 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278465    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278468    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278470    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278473    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278475    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278479    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278481    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278484    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:04.278486    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.278554    2574 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.281821    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.282591    2574 server.go:1014] "Starting client certificate rotation"
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.282698    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.282745    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.284759    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:33:04.289051 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.285945    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:33:04.295292 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.295275    2574 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:33:04.298022 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.298008    2574 log.go:25] "Validated CRI v1 image API"
Oct 30 21:33:04.298842 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.298822    2574 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:33:04.301146 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.301127    2574 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a7bf4f2f-778c-463f-afb8-1559d8ade28e:/dev/nvme0n1p4 ed8203bc-0c01-4572-bc85-c71eccdc6543:/dev/nvme0n1p3]
Oct 30 21:33:04.301225 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.301145    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 21:33:04.307599 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.307478    2574 manager.go:217] Machine: {Timestamp:2026-10-30 21:33:04.306318403 +0000 UTC m=+0.307370514 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100580 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec298737e412671fd1551da30cfd24af SystemUUID:ec298737-e412-671f-d155-1da30cfd24af BootID:5e6d313a-c2ee-4706-9cbe-228c25cd19f4 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:44:be:d7:26:6b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:44:be:d7:26:6b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:b6:f4:09:c1:93 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:33:04.307599 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.307597    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:33:04.307700 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.307679    2574 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:33:04.309441 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.309419    2574 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:33:04.310720 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.309442    2574 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-104.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:33:04.310720 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.310086    2574 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:33:04.310720 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.310108    2574 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:33:04.310720 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.310164    2574 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:33:04.310720 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.310208    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:33:04.311333 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.311316    2574 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:33:04.315440 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.315428    2574 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:33:04.315560 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.315551    2574 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:33:04.316117 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.316107    2574 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:33:04.316149 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.316124    2574 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:33:04.316149 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.316135    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:33:04.316149 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.316145    2574 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:33:04.316236 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.316155    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:33:04.316512 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.316500    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:33:04.319007 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.318989    2574 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:33:04.323810 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.323796    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:33:04.324422 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.324410    2574 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:33:04.324455 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.324440    2574 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:33:04.324630 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.324617    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:33:04.324672 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.324640    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:33:04.324672 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.324650    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:33:04.324672 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.324658    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:33:04.324672 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.324666    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:33:04.324672 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.324673    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:33:04.325051 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.325043    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:33:04.325082 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.325052    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:33:04.325082 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.325060    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:33:04.325082 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.325065    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:33:04.325082 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.325074    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:33:04.325082 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.325083    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:33:04.325422 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.325413    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:33:04.325422 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.325422    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:33:04.329039 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.329027    2574 server.go:1263] "Started kubelet"
Oct 30 21:33:04.329176 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.329120    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:33:04.329229 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.329214    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:33:04.329272 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.329175    2574 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:33:04.329896 ip-10-0-130-104 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:33:04.331302 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.331281    2574 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:33:04.332703 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.332690    2574 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:33:04.334405 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:04.334384    2574 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 21:33:04.336443 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.336425    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:33:04.337376 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.337363    2574 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:33:04.337478 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:04.337464    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-104.us-west-1.compute.internal\" not found"
Oct 30 21:33:04.338128 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.338109    2574 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:33:04.338128 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.338109    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:33:04.338128 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.338131    2574 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:33:04.338318 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.338220    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:33:04.338318 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.338229    2574 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:33:04.338318 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.338235    2574 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:33:04.338318 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.338234    2574 factory.go:55] Registering systemd factory
Oct 30 21:33:04.338318 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.338268    2574 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:33:04.338561 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.338548    2574 factory.go:153] Registering CRI-O factory
Oct 30 21:33:04.338601 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.338563    2574 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:33:04.338601 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.338576    2574 factory.go:103] Registering Raw factory
Oct 30 21:33:04.338601 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.338588    2574 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:33:04.339493 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.339475    2574 manager.go:319] Starting recovery of all containers
Oct 30 21:33:04.352266 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.352245    2574 manager.go:324] Recovery completed
Oct 30 21:33:04.356470 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.356456    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:33:04.358172 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.358150    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:33:04.358231 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.358188    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:33:04.358231 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.358199    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:33:04.358693 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.358676    2574 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:33:04.358762 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.358693    2574 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:33:04.358762 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.358714    2574 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:33:04.358762 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.358752    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:33:04.359512 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.359498    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:33:04.359574 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.359530    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:33:04.359574 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.359540    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:33:04.360996 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.360980    2574 policy_none.go:50] "Start"
Oct 30 21:33:04.361073 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.361000    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:33:04.361073 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.361013    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:33:04.363074 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.363062    2574 policy_none.go:44] "Start"
Oct 30 21:33:04.385527 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.385486    2574 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:33:04.388440 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.388424    2574 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:33:04.388532 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:04.388469    2574 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:33:04.388532 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.388482    2574 server.go:87] "Starting device plugin registration server"
Oct 30 21:33:04.388785 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.388767    2574 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:33:04.388867 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.388784    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:33:04.388942 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.388921    2574 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:33:04.389029 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.389016    2574 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:33:04.389072 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.389028    2574 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:33:04.389555 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:04.389536    2574 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:33:04.389627 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:04.389589    2574 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-104.us-west-1.compute.internal\" not found"
Oct 30 21:33:04.471194 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.471159    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-104.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 21:33:04.471194 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:04.471166    2574 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-104.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 21:33:04.488942 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.488918    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:33:04.490265 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.490172    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:33:04.490265 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.490209    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:33:04.490265 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.490226    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:33:04.490403 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.490273    2574 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.527366 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:04.519668    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-104.us-west-1.compute.internal.18e36c255b8abffe  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-104.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-104.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 21:33:04.32899891 +0000 UTC m=+0.330051020,LastTimestamp:2026-10-30 21:33:04.32899891 +0000 UTC m=+0.330051020,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-104.us-west-1.compute.internal,}"
Oct 30 21:33:04.553148 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.553111    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:33:04.554338 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.554317    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:33:04.554451 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.554344    2574 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:33:04.554451 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.554369    2574 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:33:04.554451 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:04.554406    2574 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 21:33:04.557333 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.557310    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:33:04.557434 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.557368    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ftxq5"
Oct 30 21:33:04.650463 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.650424    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:33:04.651710 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.651694    2574 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:33:04.654747 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.654728    2574 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"]
Oct 30 21:33:04.654798 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.654787    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:33:04.656024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.656009    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:33:04.656100 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.656038    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:33:04.656100 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.656050    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:33:04.658405 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.658394    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:33:04.659019 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.659002    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.659110 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.659034    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:33:04.659151 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.659124    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:33:04.659189 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.659154    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:33:04.659189 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.659165    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:33:04.659715 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.659703    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:33:04.659757 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.659726    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:33:04.659757 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.659737    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:33:04.661226 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.661204    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.661308 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.661233    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:33:04.661920 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.661894    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:33:04.661996 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.661936    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:33:04.661996 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.661946    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:33:04.690617 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:04.690598    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-104.us-west-1.compute.internal\" not found" node="ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.693302 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:04.693284    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-104.us-west-1.compute.internal\" not found" node="ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.740620 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.740584    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8eb924ba35b0f8556b9fd18e228cc9da-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"8eb924ba35b0f8556b9fd18e228cc9da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.741039 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.740620    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8eb924ba35b0f8556b9fd18e228cc9da-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"8eb924ba35b0f8556b9fd18e228cc9da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.741086 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.741060    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ea3e08f52bca920602c76ad35bcaf69-config\") pod \"kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"6ea3e08f52bca920602c76ad35bcaf69\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.749159 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.749138    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:33:04.750223 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.750205    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:33:04.750943 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.750928    2574 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.751274 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:04.750950    2574 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-104.us-west-1.compute.internal\": node \"ip-10-0-130-104.us-west-1.compute.internal\" not found"
Oct 30 21:33:04.838637 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.838559    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.841875 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.841850    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8eb924ba35b0f8556b9fd18e228cc9da-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"8eb924ba35b0f8556b9fd18e228cc9da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.843579 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.841734    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8eb924ba35b0f8556b9fd18e228cc9da-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"8eb924ba35b0f8556b9fd18e228cc9da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.843579 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.842688    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8eb924ba35b0f8556b9fd18e228cc9da-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"8eb924ba35b0f8556b9fd18e228cc9da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.843579 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.842777    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ea3e08f52bca920602c76ad35bcaf69-config\") pod \"kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"6ea3e08f52bca920602c76ad35bcaf69\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.843579 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.842857    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ea3e08f52bca920602c76ad35bcaf69-config\") pod \"kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"6ea3e08f52bca920602c76ad35bcaf69\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.843579 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.842947    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8eb924ba35b0f8556b9fd18e228cc9da-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"8eb924ba35b0f8556b9fd18e228cc9da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.898009 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.897979    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:33:04.898169 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.898102    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.955666 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.955634    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:33:04.993822 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.993789    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:04.995937 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:04.995902    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:33:06.893191 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:06.893020    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 21:33:06.893672 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:06.893381    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:33:06.893672 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:06.893381    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:33:06.893672 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:06.893393    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:33:06.893672 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:06.893437    2574 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a87acbcee0c914c0587855f827b89858-cda6e39977984c9a.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-130-104.us-west-1.compute.internal.18e36c255d485c51\": read tcp 10.0.130.104:38618->18.190.185.126:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-130-104.us-west-1.compute.internal.18e36c255d485c51  default   5423 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-104.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-104.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-104.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 21:33:04 +0000 UTC,LastTimestamp:2026-10-30 21:33:04.490232406 +0000 UTC m=+0.491284518,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-104.us-west-1.compute.internal,}"
Oct 30 21:33:06.927322 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:06.927283    2574 apiserver.go:52] "Watching apiserver"
Oct 30 21:33:06.947173 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:06.947147    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 21:33:07.174708 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.174611    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:28:06 +0000 UTC" deadline="2028-05-24 10:33:19.121394477 +0000 UTC"
Oct 30 21:33:07.174708 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.174647    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13717h0m11.94674994s"
Oct 30 21:33:07.198091 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.198055    2574 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:33:07.198392 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.198369    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-jfbvs","kube-system/global-pull-secret-syncer-ztncc","kube-system/konnectivity-agent-9zq7q","kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px","openshift-multus/multus-2m72k","openshift-multus/multus-additional-cni-plugins-kx4s8","openshift-cluster-node-tuning-operator/tuned-xrkmw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal","openshift-multus/network-metrics-daemon-w9rmz","openshift-dns/node-resolver-z8g8q","openshift-image-registry/node-ca-jgphd","openshift-network-diagnostics/network-check-target-2r74t","openshift-network-operator/iptables-alerter-wdcq5"]
Oct 30 21:33:07.200936 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.200920    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.202953 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.202935    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:07.203064 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.202994    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:07.204989 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.204973    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:33:07.207091 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.207074    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.209129 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.209114    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.211338 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.211316    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.213620 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.213603    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.215919 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.215900    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:07.216004 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.215972    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:07.219005 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.218885    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:33:07.221294 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.221277    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:33:07.223569 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.223549    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:07.223669 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.223628    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:07.225708 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.225693    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:33:07.250089 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.250067    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:33:07.254443 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.254416    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:33:07.254769 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.254749    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:33:07.254862 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.254783    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:33:07.254862 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.254824    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:33:07.254862 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.254869    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:33:07.255063 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.254980    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:33:07.255063 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.254977    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tvlkp\""
Oct 30 21:33:07.256202 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256186    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysconfig\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.256273 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256209    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-var-lib-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.256273 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256241    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3142d576-19ca-48d1-b50f-c22945af6ba3-agent-certs\") pod \"konnectivity-agent-9zq7q\" (UID: \"3142d576-19ca-48d1-b50f-c22945af6ba3\") " pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:33:07.256273 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256259    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2ccf87bf-a091-4f22-8811-02dcf621e556-iptables-alerter-script\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:33:07.256372 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256274    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-socket-dir-parent\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.256372 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256327    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-conf-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.256429 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256376    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-host\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.256465 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256431    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-systemd-units\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.256498 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256459    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-etc-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.256498 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256483    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-log-socket\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.256554 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256511    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.256554 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256538    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx4xs\" (UniqueName: \"kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs\") pod \"network-check-target-2r74t\" (UID: \"5c24d279-0f41-4c11-b197-c7c9ce7ed028\") " pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:07.256611 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256557    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-netns\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.256611 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256577    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-cni-multus\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.256611 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256604    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9mvp\" (UniqueName: \"kubernetes.io/projected/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-kube-api-access-p9mvp\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.256705 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256619    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-kubelet\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.256705 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256643    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.256705 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256666    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.256705 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256691    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5rb7\" (UniqueName: \"kubernetes.io/projected/2ccf87bf-a091-4f22-8811-02dcf621e556-kube-api-access-h5rb7\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:33:07.256853 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256706    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-cni-bin\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.256853 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256736    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.256853 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256757    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-env-overrides\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.256853 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256797    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmwdl\" (UniqueName: \"kubernetes.io/projected/c5deaa23-062e-431f-bca9-ffb4b60159cd-kube-api-access-kmwdl\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.256853 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256817    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ccf87bf-a091-4f22-8811-02dcf621e556-host-slash\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:33:07.256853 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256831    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysctl-conf\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.257023 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256857    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-tuned\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.257023 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256883    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06512914-5083-4ad9-a3da-a6b916f53b6e-host\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:33:07.257023 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256902    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-ovn\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.257023 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256933    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovn-node-metrics-cert\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.257023 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256953    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s46xr\" (UniqueName: \"kubernetes.io/projected/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-kube-api-access-s46xr\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:33:07.257023 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.256979    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-system-cni-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.257023 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257000    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-systemd\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.257023 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257015    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c246b\" (UniqueName: \"kubernetes.io/projected/6ad46264-9367-4a4e-8eb9-eb0d0470a669-kube-api-access-c246b\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:07.257340 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257042    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-slash\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.257340 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257081    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-run-netns\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.257340 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257111    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-tmp-dir\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:33:07.257340 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257137    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-cnibin\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.257340 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257153    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-hostroot\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.257340 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257177    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk97l\" (UniqueName: \"kubernetes.io/projected/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-kube-api-access-wk97l\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.257340 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257210    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:07.257340 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257242    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-hosts-file\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:33:07.257340 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257263    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-registration-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.257340 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257280    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-k8s-cni-cncf-io\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.257340 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257308    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-multus-certs\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.257340 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257335    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-kubernetes\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257360    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-run\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257388    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbc9d\" (UniqueName: \"kubernetes.io/projected/06512914-5083-4ad9-a3da-a6b916f53b6e-kube-api-access-rbc9d\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257410    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-systemd\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257456    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90f49eda-5385-490c-9760-2cf569eabccf-cni-binary-copy\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257498    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257503    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h8w4\" (UniqueName: \"kubernetes.io/projected/90f49eda-5385-490c-9760-2cf569eabccf-kube-api-access-4h8w4\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257534    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3142d576-19ca-48d1-b50f-c22945af6ba3-konnectivity-ca\") pod \"konnectivity-agent-9zq7q\" (UID: \"3142d576-19ca-48d1-b50f-c22945af6ba3\") " pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257567    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-etc-kubernetes\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257594    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cnibin\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257614    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257630    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/06512914-5083-4ad9-a3da-a6b916f53b6e-serviceca\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257659    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovnkube-config\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257686    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovnkube-script-lib\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257708    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-lib-modules\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257725    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-socket-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257739    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-kubelet\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.257768 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257759    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-system-cni-dir\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257789    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-modprobe-d\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257811    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257828    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-device-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257858    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-etc-selinux\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257877    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257891    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-var-lib-kubelet\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257915    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-cni-bin\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257922    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257936    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fde4a00d-016e-4670-a530-f61d1e193a9c-dbus\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257973    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vplh\" (UniqueName: \"kubernetes.io/projected/e83d89de-6a2a-4950-a82a-d0705739ce81-kube-api-access-9vplh\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.257984    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-wn5fv\""
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.258007    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysctl-d\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.258036    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-tmp\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.258066    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-node-log\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.258095    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-cni-netd\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.258118    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fde4a00d-016e-4670-a530-f61d1e193a9c-kubelet-config\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:07.258395 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.258143    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:07.258981 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.258165    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-cni-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.258981 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.258193    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90f49eda-5385-490c-9760-2cf569eabccf-multus-daemon-config\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.258981 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.258218    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-sys\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.258981 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.258274    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-sys-fs\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.258981 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.258295    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.258981 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.258547    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:33:07.258981 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.258678    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:33:07.258981 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.258696    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:33:07.299809 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.299787    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pplxq\""
Oct 30 21:33:07.299911 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.299831    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:33:07.304272 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.304244    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-zs2mb\""
Oct 30 21:33:07.304470 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.304446    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:33:07.304537 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.304493    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:33:07.304578 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.304553    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6t698\""
Oct 30 21:33:07.304653 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.304636    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jnpcj\""
Oct 30 21:33:07.304949 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.304938    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:33:07.305016 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.304998    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:33:07.305073 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.305053    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mzgn4\""
Oct 30 21:33:07.305127 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.305068    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:33:07.305127 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.305082    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:33:07.305127 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.305091    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:33:07.305127 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.305114    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:33:07.305127 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.305125    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:33:07.305337 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.305139    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:33:07.305337 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.305164    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:33:07.305337 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.305247    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rjmnn\""
Oct 30 21:33:07.307066 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.307051    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:33:07.307630 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.307617    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:33:07.307738 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.307727    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2v7jg\""
Oct 30 21:33:07.307882 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.307872    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:33:07.307960 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.307947    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:33:07.308468 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.308453    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:33:07.308519 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.308505    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-srsnk"
Oct 30 21:33:07.358512 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358481    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-var-lib-kubelet\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.358512 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358514    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-cni-bin\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.358729 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358531    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fde4a00d-016e-4670-a530-f61d1e193a9c-dbus\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:07.358729 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358550    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vplh\" (UniqueName: \"kubernetes.io/projected/e83d89de-6a2a-4950-a82a-d0705739ce81-kube-api-access-9vplh\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.358729 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358585    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-cni-bin\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.358729 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358588    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-var-lib-kubelet\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.358729 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358614    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysctl-d\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.358729 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358654    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-tmp\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.358729 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358659    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fde4a00d-016e-4670-a530-f61d1e193a9c-dbus\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:07.358729 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358675    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-node-log\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.358729 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358695    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-node-log\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.358729 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358708    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-cni-netd\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358738    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fde4a00d-016e-4670-a530-f61d1e193a9c-kubelet-config\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358756    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-cni-netd\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358764    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358779    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fde4a00d-016e-4670-a530-f61d1e193a9c-kubelet-config\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358741    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysctl-d\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358788    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-cni-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358829    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90f49eda-5385-490c-9760-2cf569eabccf-multus-daemon-config\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358840    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-cni-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358851    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-sys\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358874    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-sys-fs\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358896    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358918    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-sys\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.358829    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358957    2574 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358931    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-sys-fs\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.359047    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret podName:fde4a00d-016e-4670-a530-f61d1e193a9c nodeName:}" failed. No retries permitted until 2026-10-30 21:33:07.859010818 +0000 UTC m=+2.249736412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret") pod "global-pull-secret-syncer-ztncc" (UID: "fde4a00d-016e-4670-a530-f61d1e193a9c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:07.359094 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358920    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysconfig\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359077    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-var-lib-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359094    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3142d576-19ca-48d1-b50f-c22945af6ba3-agent-certs\") pod \"konnectivity-agent-9zq7q\" (UID: \"3142d576-19ca-48d1-b50f-c22945af6ba3\") " pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359111    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2ccf87bf-a091-4f22-8811-02dcf621e556-iptables-alerter-script\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359137    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-socket-dir-parent\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.358949    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysconfig\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359156    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-var-lib-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359171    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-conf-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359195    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-socket-dir-parent\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359196    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-host\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359227    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-conf-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359258    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-systemd-units\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359275    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-host\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359286    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-etc-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359318    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-systemd-units\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359310    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-log-socket\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359362    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-etc-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359371    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-log-socket\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.359910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359395    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90f49eda-5385-490c-9760-2cf569eabccf-multus-daemon-config\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359388    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359433    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359439    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx4xs\" (UniqueName: \"kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs\") pod \"network-check-target-2r74t\" (UID: \"5c24d279-0f41-4c11-b197-c7c9ce7ed028\") " pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359458    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359463    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-netns\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359481    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-netns\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359509    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-cni-multus\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359530    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9mvp\" (UniqueName: \"kubernetes.io/projected/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-kube-api-access-p9mvp\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359550    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-kubelet\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359571    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359593    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359582    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-cni-multus\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359617    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5rb7\" (UniqueName: \"kubernetes.io/projected/2ccf87bf-a091-4f22-8811-02dcf621e556-kube-api-access-h5rb7\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359638    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-cni-bin\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359652    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359660    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.360723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359678    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359683    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-kubelet\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359682    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-env-overrides\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359719    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmwdl\" (UniqueName: \"kubernetes.io/projected/c5deaa23-062e-431f-bca9-ffb4b60159cd-kube-api-access-kmwdl\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359729    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2ccf87bf-a091-4f22-8811-02dcf621e556-iptables-alerter-script\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359745    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ccf87bf-a091-4f22-8811-02dcf621e556-host-slash\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359768    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ccf87bf-a091-4f22-8811-02dcf621e556-host-slash\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359787    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysctl-conf\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359806    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-cni-bin\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359821    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-tuned\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359846    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06512914-5083-4ad9-a3da-a6b916f53b6e-host\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359869    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-ovn\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359900    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovn-node-metrics-cert\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359918    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysctl-conf\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359925    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s46xr\" (UniqueName: \"kubernetes.io/projected/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-kube-api-access-s46xr\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359959    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-ovn\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.359996    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06512914-5083-4ad9-a3da-a6b916f53b6e-host\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.360037    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-env-overrides\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.361547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.360064    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.360276    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-system-cni-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.360419    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-system-cni-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.360457    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-systemd\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.360483    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c246b\" (UniqueName: \"kubernetes.io/projected/6ad46264-9367-4a4e-8eb9-eb0d0470a669-kube-api-access-c246b\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.360508    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-slash\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.360533    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-run-netns\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.360569    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-systemd\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.360877    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-tmp-dir\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.360927    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-cnibin\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.360962    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-hostroot\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.360997    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk97l\" (UniqueName: \"kubernetes.io/projected/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-kube-api-access-wk97l\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361033    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361073    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-hosts-file\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361104    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-registration-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361138    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-k8s-cni-cncf-io\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361170    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-multus-certs\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361201    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-kubernetes\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.362377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361227    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-run\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361283    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbc9d\" (UniqueName: \"kubernetes.io/projected/06512914-5083-4ad9-a3da-a6b916f53b6e-kube-api-access-rbc9d\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361314    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-systemd\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361327    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-slash\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361349    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90f49eda-5385-490c-9760-2cf569eabccf-cni-binary-copy\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361417    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h8w4\" (UniqueName: \"kubernetes.io/projected/90f49eda-5385-490c-9760-2cf569eabccf-kube-api-access-4h8w4\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361473    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3142d576-19ca-48d1-b50f-c22945af6ba3-konnectivity-ca\") pod \"konnectivity-agent-9zq7q\" (UID: \"3142d576-19ca-48d1-b50f-c22945af6ba3\") " pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361477    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-run-netns\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361522    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-etc-kubernetes\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361581    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cnibin\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361616    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361647    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/06512914-5083-4ad9-a3da-a6b916f53b6e-serviceca\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361680    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovnkube-config\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361707    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovnkube-script-lib\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361750    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-lib-modules\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.361777    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-socket-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.362024    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-socket-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.362349    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-tmp-dir\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:33:07.363157 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.362481    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90f49eda-5385-490c-9760-2cf569eabccf-cni-binary-copy\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.362538    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-kubelet\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.362589    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-system-cni-dir\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.362614    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-cnibin\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.362638    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-modprobe-d\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.362675    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.362684    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-hostroot\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.362736    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-device-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.362855    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-device-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.362908    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-system-cni-dir\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.362943    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-tuned\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.362970    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-kubelet\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363027    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-systemd\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363027    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovn-node-metrics-cert\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.363139    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363145    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-modprobe-d\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363172    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.363970 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.363353    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs podName:6ad46264-9367-4a4e-8eb9-eb0d0470a669 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:07.863333596 +0000 UTC m=+2.254059181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs") pod "network-metrics-daemon-w9rmz" (UID: "6ad46264-9367-4a4e-8eb9-eb0d0470a669") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:07.364694 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363460    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-etc-kubernetes\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.364694 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363462    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-multus-certs\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.364694 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363534    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cnibin\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.364694 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363591    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-kubernetes\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.364694 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363668    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-run\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.364694 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363720    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-k8s-cni-cncf-io\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.364694 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363642    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3142d576-19ca-48d1-b50f-c22945af6ba3-konnectivity-ca\") pod \"konnectivity-agent-9zq7q\" (UID: \"3142d576-19ca-48d1-b50f-c22945af6ba3\") " pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:33:07.364694 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363775    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-etc-selinux\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.364694 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363811    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.364694 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363931    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-hosts-file\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:33:07.364694 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363935    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-lib-modules\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.364694 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.363996    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-etc-selinux\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.364694 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.364102    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-registration-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.364694 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.364627    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.366331 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.366305    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-tmp\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.366563 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.366543    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3142d576-19ca-48d1-b50f-c22945af6ba3-agent-certs\") pod \"konnectivity-agent-9zq7q\" (UID: \"3142d576-19ca-48d1-b50f-c22945af6ba3\") " pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:33:07.366800 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.366772    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovnkube-script-lib\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.367015 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.366992    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/06512914-5083-4ad9-a3da-a6b916f53b6e-serviceca\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:33:07.367015 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.367007    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovnkube-config\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.367015 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.367009    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.415597 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.415566    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:33:07.415597 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.415594    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:33:07.415597 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.415607    2574 projected.go:196] Error preparing data for projected volume kube-api-access-fx4xs for pod openshift-network-diagnostics/network-check-target-2r74t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:07.416651 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.416616    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs podName:5c24d279-0f41-4c11-b197-c7c9ce7ed028 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:07.916588778 +0000 UTC m=+2.307314381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fx4xs" (UniqueName: "kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs") pod "network-check-target-2r74t" (UID: "5c24d279-0f41-4c11-b197-c7c9ce7ed028") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:07.418327 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.418292    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmwdl\" (UniqueName: \"kubernetes.io/projected/c5deaa23-062e-431f-bca9-ffb4b60159cd-kube-api-access-kmwdl\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.418440 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.418387    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s46xr\" (UniqueName: \"kubernetes.io/projected/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-kube-api-access-s46xr\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:33:07.418516 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.418451    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vplh\" (UniqueName: \"kubernetes.io/projected/e83d89de-6a2a-4950-a82a-d0705739ce81-kube-api-access-9vplh\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.418572 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.418525    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9mvp\" (UniqueName: \"kubernetes.io/projected/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-kube-api-access-p9mvp\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.419078 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.419056    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5rb7\" (UniqueName: \"kubernetes.io/projected/2ccf87bf-a091-4f22-8811-02dcf621e556-kube-api-access-h5rb7\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:33:07.420073 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.420052    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c246b\" (UniqueName: \"kubernetes.io/projected/6ad46264-9367-4a4e-8eb9-eb0d0470a669-kube-api-access-c246b\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:07.420552 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.420535    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbc9d\" (UniqueName: \"kubernetes.io/projected/06512914-5083-4ad9-a3da-a6b916f53b6e-kube-api-access-rbc9d\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:33:07.420769 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.420750    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h8w4\" (UniqueName: \"kubernetes.io/projected/90f49eda-5385-490c-9760-2cf569eabccf-kube-api-access-4h8w4\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.421245 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.421216    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk97l\" (UniqueName: \"kubernetes.io/projected/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-kube-api-access-wk97l\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.535134 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.535055    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:07.538193 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.538161    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:33:07.553276 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.553253    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:33:07.559477 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.559457    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2m72k"
Oct 30 21:33:07.581317 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.581285    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:33:07.585214 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.585194    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:33:07.591804 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.591783    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:33:07.609508 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.609476    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:33:07.614641 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.614610    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:33:07.716971 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:07.716931    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ea3e08f52bca920602c76ad35bcaf69.slice/crio-dcbfb3ad0699a53006ea978e770fc8ccbd4d48fe7df3751603bb15d74fa4de82 WatchSource:0}: Error finding container dcbfb3ad0699a53006ea978e770fc8ccbd4d48fe7df3751603bb15d74fa4de82: Status 404 returned error can't find the container with id dcbfb3ad0699a53006ea978e770fc8ccbd4d48fe7df3751603bb15d74fa4de82
Oct 30 21:33:07.717362 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:07.717343    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eb924ba35b0f8556b9fd18e228cc9da.slice/crio-1414efc0789b873d1e4beb482d7086f82ea661b7b8fe156ee1808d885f1ea5db WatchSource:0}: Error finding container 1414efc0789b873d1e4beb482d7086f82ea661b7b8fe156ee1808d885f1ea5db: Status 404 returned error can't find the container with id 1414efc0789b873d1e4beb482d7086f82ea661b7b8fe156ee1808d885f1ea5db
Oct 30 21:33:07.723050 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.723030    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:33:07.868360 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.868280    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:07.868360 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.868345    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:07.868542 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.868433    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:07.868542 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.868493    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret podName:fde4a00d-016e-4670-a530-f61d1e193a9c nodeName:}" failed. No retries permitted until 2026-10-30 21:33:08.868479598 +0000 UTC m=+3.259205183 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret") pod "global-pull-secret-syncer-ztncc" (UID: "fde4a00d-016e-4670-a530-f61d1e193a9c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:07.868542 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.868433    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:07.868542 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.868537    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs podName:6ad46264-9367-4a4e-8eb9-eb0d0470a669 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:08.868526863 +0000 UTC m=+3.259252458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs") pod "network-metrics-daemon-w9rmz" (UID: "6ad46264-9367-4a4e-8eb9-eb0d0470a669") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:07.948368 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.948334    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:33:07.968826 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:07.968788    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx4xs\" (UniqueName: \"kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs\") pod \"network-check-target-2r74t\" (UID: \"5c24d279-0f41-4c11-b197-c7c9ce7ed028\") " pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:07.969070 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.968936    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:33:07.969070 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.968955    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:33:07.969070 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.968966    2574 projected.go:196] Error preparing data for projected volume kube-api-access-fx4xs for pod openshift-network-diagnostics/network-check-target-2r74t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:07.969070 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:07.969037    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs podName:5c24d279-0f41-4c11-b197-c7c9ce7ed028 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:08.969019086 +0000 UTC m=+3.359744666 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fx4xs" (UniqueName: "kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs") pod "network-check-target-2r74t" (UID: "5c24d279-0f41-4c11-b197-c7c9ce7ed028") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:08.169148 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:08.169048    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal" event={"ID":"8eb924ba35b0f8556b9fd18e228cc9da","Type":"ContainerStarted","Data":"1414efc0789b873d1e4beb482d7086f82ea661b7b8fe156ee1808d885f1ea5db"}
Oct 30 21:33:08.170085 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:08.170058    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal" event={"ID":"6ea3e08f52bca920602c76ad35bcaf69","Type":"ContainerStarted","Data":"dcbfb3ad0699a53006ea978e770fc8ccbd4d48fe7df3751603bb15d74fa4de82"}
Oct 30 21:33:08.311635 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:08.311599    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:28:07 +0000 UTC" deadline="2028-06-11 08:05:22.369343032 +0000 UTC"
Oct 30 21:33:08.311635 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:08.311627    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14146h32m14.057719328s"
Oct 30 21:33:08.362733 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:08.362702    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90f49eda_5385_490c_9760_2cf569eabccf.slice/crio-5cbd08bc248d2f9c204a3db6c4a799d7ddeafe5e70f3f4527425fb73b9223c1c WatchSource:0}: Error finding container 5cbd08bc248d2f9c204a3db6c4a799d7ddeafe5e70f3f4527425fb73b9223c1c: Status 404 returned error can't find the container with id 5cbd08bc248d2f9c204a3db6c4a799d7ddeafe5e70f3f4527425fb73b9223c1c
Oct 30 21:33:08.405974 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:08.405930    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:33:08.449267 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:08.449223    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaebf7fc_ea88_4bb8_aad6_d9ae838fe5b8.slice/crio-df91d26b206e05676cd3152cc7a2bedf8c077a01298b4d42b3d273d3834ea190 WatchSource:0}: Error finding container df91d26b206e05676cd3152cc7a2bedf8c077a01298b4d42b3d273d3834ea190: Status 404 returned error can't find the container with id df91d26b206e05676cd3152cc7a2bedf8c077a01298b4d42b3d273d3834ea190
Oct 30 21:33:08.464385 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:08.464356    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32dd7eaa_b8a6_4de5_a13a_0640b94801e8.slice/crio-b36bb7162e2a9d563a8b6577e801656400e153b70272903fe2b9f085420cae77 WatchSource:0}: Error finding container b36bb7162e2a9d563a8b6577e801656400e153b70272903fe2b9f085420cae77: Status 404 returned error can't find the container with id b36bb7162e2a9d563a8b6577e801656400e153b70272903fe2b9f085420cae77
Oct 30 21:33:08.468474 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:08.468134    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3142d576_19ca_48d1_b50f_c22945af6ba3.slice/crio-14cda20d61c1b9e007361243c4d97487f5dd4de58cc27565b4c4d816dbe5ab4c WatchSource:0}: Error finding container 14cda20d61c1b9e007361243c4d97487f5dd4de58cc27565b4c4d816dbe5ab4c: Status 404 returned error can't find the container with id 14cda20d61c1b9e007361243c4d97487f5dd4de58cc27565b4c4d816dbe5ab4c
Oct 30 21:33:08.468861 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:08.468800    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ccf87bf_a091_4f22_8811_02dcf621e556.slice/crio-33ff5cbba2f296cb9e6a579230ee4feb6697a7e90788051c60d95896bd6e5a6a WatchSource:0}: Error finding container 33ff5cbba2f296cb9e6a579230ee4feb6697a7e90788051c60d95896bd6e5a6a: Status 404 returned error can't find the container with id 33ff5cbba2f296cb9e6a579230ee4feb6697a7e90788051c60d95896bd6e5a6a
Oct 30 21:33:08.472213 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:08.472188    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:33:08.503666 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:08.503629    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode83d89de_6a2a_4950_a82a_d0705739ce81.slice/crio-d7cad3ecd7cbdf5061c9f000c7754e145c57d802ba7dac726736f93ac1a2f7dd WatchSource:0}: Error finding container d7cad3ecd7cbdf5061c9f000c7754e145c57d802ba7dac726736f93ac1a2f7dd: Status 404 returned error can't find the container with id d7cad3ecd7cbdf5061c9f000c7754e145c57d802ba7dac726736f93ac1a2f7dd
Oct 30 21:33:08.523682 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:08.523648    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5deaa23_062e_431f_bca9_ffb4b60159cd.slice/crio-a7fde4cea4342ec8f9c95988990190640238b5389c9ea67da85d41f8364a95c4 WatchSource:0}: Error finding container a7fde4cea4342ec8f9c95988990190640238b5389c9ea67da85d41f8364a95c4: Status 404 returned error can't find the container with id a7fde4cea4342ec8f9c95988990190640238b5389c9ea67da85d41f8364a95c4
Oct 30 21:33:08.524563 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:08.524531    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06512914_5083_4ad9_a3da_a6b916f53b6e.slice/crio-9c2c8b2b422eca5331baa11b3b69649a1681e5ffe0a5df99f9fe2dd8517588df WatchSource:0}: Error finding container 9c2c8b2b422eca5331baa11b3b69649a1681e5ffe0a5df99f9fe2dd8517588df: Status 404 returned error can't find the container with id 9c2c8b2b422eca5331baa11b3b69649a1681e5ffe0a5df99f9fe2dd8517588df
Oct 30 21:33:08.876214 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:08.876131    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:08.876214 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:08.876195    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:08.876463 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:08.876331    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:08.876463 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:08.876395    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret podName:fde4a00d-016e-4670-a530-f61d1e193a9c nodeName:}" failed. No retries permitted until 2026-10-30 21:33:10.876378779 +0000 UTC m=+5.267104376 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret") pod "global-pull-secret-syncer-ztncc" (UID: "fde4a00d-016e-4670-a530-f61d1e193a9c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:08.876463 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:08.876460    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:08.876563 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:08.876492    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs podName:6ad46264-9367-4a4e-8eb9-eb0d0470a669 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:10.876480562 +0000 UTC m=+5.267206157 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs") pod "network-metrics-daemon-w9rmz" (UID: "6ad46264-9367-4a4e-8eb9-eb0d0470a669") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:08.976524 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:08.976484    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx4xs\" (UniqueName: \"kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs\") pod \"network-check-target-2r74t\" (UID: \"5c24d279-0f41-4c11-b197-c7c9ce7ed028\") " pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:08.977122 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:08.976651    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:33:08.977122 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:08.976673    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:33:08.977122 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:08.976685    2574 projected.go:196] Error preparing data for projected volume kube-api-access-fx4xs for pod openshift-network-diagnostics/network-check-target-2r74t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:08.977122 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:08.976735    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs podName:5c24d279-0f41-4c11-b197-c7c9ce7ed028 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:10.976721004 +0000 UTC m=+5.367446602 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fx4xs" (UniqueName: "kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs") pod "network-check-target-2r74t" (UID: "5c24d279-0f41-4c11-b197-c7c9ce7ed028") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:09.165953 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:09.165864    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:09.165953 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:09.165902    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:09.166188 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:09.165884    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:09.166188 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:09.166021    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:09.166188 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:09.166093    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:09.166188 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:09.166181    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:09.173549 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:09.173518    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z8g8q" event={"ID":"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8","Type":"ContainerStarted","Data":"df91d26b206e05676cd3152cc7a2bedf8c077a01298b4d42b3d273d3834ea190"}
Oct 30 21:33:09.174916 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:09.174890    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2m72k" event={"ID":"90f49eda-5385-490c-9760-2cf569eabccf","Type":"ContainerStarted","Data":"5cbd08bc248d2f9c204a3db6c4a799d7ddeafe5e70f3f4527425fb73b9223c1c"}
Oct 30 21:33:09.176308 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:09.176284    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jgphd" event={"ID":"06512914-5083-4ad9-a3da-a6b916f53b6e","Type":"ContainerStarted","Data":"9c2c8b2b422eca5331baa11b3b69649a1681e5ffe0a5df99f9fe2dd8517588df"}
Oct 30 21:33:09.177810 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:09.177561    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px" event={"ID":"e83d89de-6a2a-4950-a82a-d0705739ce81","Type":"ContainerStarted","Data":"d7cad3ecd7cbdf5061c9f000c7754e145c57d802ba7dac726736f93ac1a2f7dd"}
Oct 30 21:33:09.178830 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:09.178808    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wdcq5" event={"ID":"2ccf87bf-a091-4f22-8811-02dcf621e556","Type":"ContainerStarted","Data":"33ff5cbba2f296cb9e6a579230ee4feb6697a7e90788051c60d95896bd6e5a6a"}
Oct 30 21:33:09.179972 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:09.179948    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9zq7q" event={"ID":"3142d576-19ca-48d1-b50f-c22945af6ba3","Type":"ContainerStarted","Data":"14cda20d61c1b9e007361243c4d97487f5dd4de58cc27565b4c4d816dbe5ab4c"}
Oct 30 21:33:09.181136 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:09.181113    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xrkmw" event={"ID":"32dd7eaa-b8a6-4de5-a13a-0640b94801e8","Type":"ContainerStarted","Data":"b36bb7162e2a9d563a8b6577e801656400e153b70272903fe2b9f085420cae77"}
Oct 30 21:33:09.182392 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:09.182340    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kx4s8" event={"ID":"3adefe8b-2ffa-4ee1-aad4-a658970e79aa","Type":"ContainerStarted","Data":"ce77db454de78f3ada0813466ade82455414969ab952277c2fc09bbef09e2a49"}
Oct 30 21:33:09.183678 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:09.183653    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs" event={"ID":"c5deaa23-062e-431f-bca9-ffb4b60159cd","Type":"ContainerStarted","Data":"a7fde4cea4342ec8f9c95988990190640238b5389c9ea67da85d41f8364a95c4"}
Oct 30 21:33:09.312735 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:09.312673    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:28:07 +0000 UTC" deadline="2028-06-27 02:00:40.471601634 +0000 UTC"
Oct 30 21:33:09.312735 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:09.312733    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14524h27m31.158872361s"
Oct 30 21:33:10.893203 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:10.893164    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:10.894122 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:10.893251    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:10.894122 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:10.893409    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:10.894122 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:10.893471    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret podName:fde4a00d-016e-4670-a530-f61d1e193a9c nodeName:}" failed. No retries permitted until 2026-10-30 21:33:14.893452612 +0000 UTC m=+9.284178192 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret") pod "global-pull-secret-syncer-ztncc" (UID: "fde4a00d-016e-4670-a530-f61d1e193a9c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:10.894122 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:10.893537    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:10.894122 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:10.893573    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs podName:6ad46264-9367-4a4e-8eb9-eb0d0470a669 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:14.893562609 +0000 UTC m=+9.284288193 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs") pod "network-metrics-daemon-w9rmz" (UID: "6ad46264-9367-4a4e-8eb9-eb0d0470a669") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:10.997540 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:10.997507    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx4xs\" (UniqueName: \"kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs\") pod \"network-check-target-2r74t\" (UID: \"5c24d279-0f41-4c11-b197-c7c9ce7ed028\") " pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:10.997540 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:10.997680    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:33:10.997540 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:10.997696    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:33:10.998012 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:10.997705    2574 projected.go:196] Error preparing data for projected volume kube-api-access-fx4xs for pod openshift-network-diagnostics/network-check-target-2r74t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:10.998012 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:10.997762    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs podName:5c24d279-0f41-4c11-b197-c7c9ce7ed028 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:14.997742793 +0000 UTC m=+9.388468381 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fx4xs" (UniqueName: "kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs") pod "network-check-target-2r74t" (UID: "5c24d279-0f41-4c11-b197-c7c9ce7ed028") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:11.166191 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:11.165553    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:11.166191 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:11.165772    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:11.166191 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:11.165896    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:11.166191 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:11.166013    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:11.166191 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:11.166069    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:11.166191 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:11.166136    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:11.210643 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:11.210529    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal" event={"ID":"6ea3e08f52bca920602c76ad35bcaf69","Type":"ContainerStarted","Data":"5793af378c29fe1e044424dd6e8703df12d4e595fb89c7fe490c34be53601f13"}
Oct 30 21:33:11.328392 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:11.328303    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal" podStartSLOduration=5.328282423 podStartE2EDuration="5.328282423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:11.325951382 +0000 UTC m=+5.716676988" watchObservedRunningTime="2026-10-30 21:33:11.328282423 +0000 UTC m=+5.719008026"
Oct 30 21:33:12.218313 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:12.218219    2574 generic.go:356] "Generic (PLEG): container finished" podID="8eb924ba35b0f8556b9fd18e228cc9da" containerID="3b266933a07282503efa61a74ae22378f84cf2681e028d7b8711e13686a0c950" exitCode=0
Oct 30 21:33:12.218776 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:12.218342    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal" event={"ID":"8eb924ba35b0f8556b9fd18e228cc9da","Type":"ContainerDied","Data":"3b266933a07282503efa61a74ae22378f84cf2681e028d7b8711e13686a0c950"}
Oct 30 21:33:12.224767 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:12.224732    2574 generic.go:356] "Generic (PLEG): container finished" podID="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" containerID="06305bf6ac59de3ec94a0092661ba39f391f56f0f31892935b3df5cb3cd7ceef" exitCode=0
Oct 30 21:33:12.225817 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:12.225785    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kx4s8" event={"ID":"3adefe8b-2ffa-4ee1-aad4-a658970e79aa","Type":"ContainerDied","Data":"06305bf6ac59de3ec94a0092661ba39f391f56f0f31892935b3df5cb3cd7ceef"}
Oct 30 21:33:13.166664 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:13.166551    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:13.166664 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:13.166570    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:13.166664 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:13.166581    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:13.166872 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:13.166689    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:13.166872 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:13.166795    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:13.166934 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:13.166878    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:13.231295 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:13.231251    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9zq7q" event={"ID":"3142d576-19ca-48d1-b50f-c22945af6ba3","Type":"ContainerStarted","Data":"b3bc6e67038799f7cf404e408078870016c9efa586a6a7f71d5ac9336a440d03"}
Oct 30 21:33:13.241008 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:13.240928    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal" event={"ID":"8eb924ba35b0f8556b9fd18e228cc9da","Type":"ContainerStarted","Data":"b2767587452130ada8015d789f303a81bcbc55b5aa1d94e2f78636fe5809e927"}
Oct 30 21:33:13.343961 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:13.343894    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9zq7q" podStartSLOduration=2.9034673399999997 podStartE2EDuration="7.343874632s" totalImagesPullingTime="4.440407292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:13.343399595 +0000 UTC m=+7.734125199" watchObservedRunningTime="2026-10-30 21:33:13.343874632 +0000 UTC m=+7.734600237"
Oct 30 21:33:14.378098 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:14.377631    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:33:14.378609 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:14.378160    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:33:14.491348 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:14.491296    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal" podStartSLOduration=8.491282985 podStartE2EDuration="8.491282985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:13.456423698 +0000 UTC m=+7.847149301" watchObservedRunningTime="2026-10-30 21:33:14.491282985 +0000 UTC m=+8.882008587"
Oct 30 21:33:14.930521 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:14.930474    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:14.930706 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:14.930574    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:14.930781 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:14.930730    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:14.930833 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:14.930799    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs podName:6ad46264-9367-4a4e-8eb9-eb0d0470a669 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:22.93077907 +0000 UTC m=+17.321504674 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs") pod "network-metrics-daemon-w9rmz" (UID: "6ad46264-9367-4a4e-8eb9-eb0d0470a669") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:14.930905 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:14.930869    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:14.930958 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:14.930908    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret podName:fde4a00d-016e-4670-a530-f61d1e193a9c nodeName:}" failed. No retries permitted until 2026-10-30 21:33:22.930897248 +0000 UTC m=+17.321622832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret") pod "global-pull-secret-syncer-ztncc" (UID: "fde4a00d-016e-4670-a530-f61d1e193a9c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:15.030924 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:15.030883    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx4xs\" (UniqueName: \"kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs\") pod \"network-check-target-2r74t\" (UID: \"5c24d279-0f41-4c11-b197-c7c9ce7ed028\") " pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:15.031091 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:15.031079    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:33:15.031156 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:15.031102    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:33:15.031156 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:15.031115    2574 projected.go:196] Error preparing data for projected volume kube-api-access-fx4xs for pod openshift-network-diagnostics/network-check-target-2r74t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:15.031285 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:15.031172    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs podName:5c24d279-0f41-4c11-b197-c7c9ce7ed028 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:23.031155203 +0000 UTC m=+17.421880797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fx4xs" (UniqueName: "kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs") pod "network-check-target-2r74t" (UID: "5c24d279-0f41-4c11-b197-c7c9ce7ed028") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:15.165173 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:15.165128    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:15.165379 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:15.165282    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:15.165379 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:15.165373    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:15.165568 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:15.165460    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:15.165568 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:15.165511    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:15.165665 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:15.165580    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:15.189039 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:15.188999    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:15.250111 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:15.250076    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jgphd" event={"ID":"06512914-5083-4ad9-a3da-a6b916f53b6e","Type":"ContainerStarted","Data":"71acaf2d11d049bf84a7ea4ac0e2252af2415ba49a01ad420570d54025183f53"}
Oct 30 21:33:15.255152 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:15.255123    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px" event={"ID":"e83d89de-6a2a-4950-a82a-d0705739ce81","Type":"ContainerStarted","Data":"c5c60a5696937b80d3494c793c165cdb6fb356609764d3d5bce02a191e29ec7c"}
Oct 30 21:33:15.369485 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:15.369367    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jgphd" podStartSLOduration=3.794593612 podStartE2EDuration="9.369345312s" totalImagesPullingTime="5.5747517s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:15.368386838 +0000 UTC m=+9.759112443" watchObservedRunningTime="2026-10-30 21:33:15.369345312 +0000 UTC m=+9.760070915"
Oct 30 21:33:15.416565 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:15.416530    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:16.256824 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:16.256791    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:33:17.068185 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:17.068139    2574 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:33:17.165497 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:17.165209    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:17.165497 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:17.165371    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:17.165497 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:17.165462    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:17.165721 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:17.165550    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:17.165721 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:17.165604    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:17.165721 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:17.165657    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:17.260728 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:17.260686    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z8g8q" event={"ID":"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8","Type":"ContainerStarted","Data":"bd34e2466e6f42eb61a5813d545a1b5409ddc9dcaf2215e7a5fccbf4b23fd1d4"}
Oct 30 21:33:17.264164 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:17.264135    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2m72k" event={"ID":"90f49eda-5385-490c-9760-2cf569eabccf","Type":"ContainerStarted","Data":"db2b643b94741b96eb3ce1ae80a3e5d7ee21db1bc50576e3538d9675eae6def5"}
Oct 30 21:33:17.267732 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:17.267701    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px" event={"ID":"e83d89de-6a2a-4950-a82a-d0705739ce81","Type":"ContainerStarted","Data":"252caa510aec3673759f6ffd9c19b76a6c34d90ce479221a8c62c2ebf5e22149"}
Oct 30 21:33:17.491779 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:17.491712    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-2m72k" podStartSLOduration=3.336551089 podStartE2EDuration="11.491689166s" totalImagesPullingTime="8.155138077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:17.491195609 +0000 UTC m=+11.881921212" watchObservedRunningTime="2026-10-30 21:33:17.491689166 +0000 UTC m=+11.882414769"
Oct 30 21:33:17.492122 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:17.492087    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-z8g8q" podStartSLOduration=3.282086901 podStartE2EDuration="11.492076139s" totalImagesPullingTime="8.209989238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:17.376284726 +0000 UTC m=+11.767010331" watchObservedRunningTime="2026-10-30 21:33:17.492076139 +0000 UTC m=+11.882801762"
Oct 30 21:33:18.006945 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:18.006823    2574 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:33:17.068165157Z","UUID":"055bf893-6e6a-488f-838c-282c83043197","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:33:18.008628 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:18.008602    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:33:18.008628 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:18.008637    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:33:18.272329 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:18.272252    2574 generic.go:356] "Generic (PLEG): container finished" podID="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" containerID="9942b46ba4e9c7adf714f41361b19b26a58462cadd887ed40fc050c1e821bcb2" exitCode=0
Oct 30 21:33:18.272329 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:18.272301    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kx4s8" event={"ID":"3adefe8b-2ffa-4ee1-aad4-a658970e79aa","Type":"ContainerDied","Data":"9942b46ba4e9c7adf714f41361b19b26a58462cadd887ed40fc050c1e821bcb2"}
Oct 30 21:33:18.630375 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:18.630185    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:18.776895 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:18.776864    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:19.165697 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:19.165663    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:19.165850 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:19.165774    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:19.165850 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:19.165780    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:19.165962 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:19.165923    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:19.165962 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:19.165951    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:19.166061 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:19.166023    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:19.277254 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:19.277147    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px" event={"ID":"e83d89de-6a2a-4950-a82a-d0705739ce81","Type":"ContainerStarted","Data":"c06e2c02537e28bfa57220813c992a2b05b260c35c88201a91709f86517e6ad8"}
Oct 30 21:33:19.278869 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:19.278842    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wdcq5" event={"ID":"2ccf87bf-a091-4f22-8811-02dcf621e556","Type":"ContainerStarted","Data":"678b36123da17b2bc23548f5021c92b62db41317c85210726b167f4ca0ef812a"}
Oct 30 21:33:19.506135 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:19.506067    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px" podStartSLOduration=3.882234508 podStartE2EDuration="13.506047589s" totalImagesPullingTime="9.623813081s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:19.392525101 +0000 UTC m=+13.783250704" watchObservedRunningTime="2026-10-30 21:33:19.506047589 +0000 UTC m=+13.896773193"
Oct 30 21:33:19.506807 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:19.506772    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wdcq5" podStartSLOduration=4.687685 podStartE2EDuration="13.506760716s" totalImagesPullingTime="8.819075716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:19.505894835 +0000 UTC m=+13.896620438" watchObservedRunningTime="2026-10-30 21:33:19.506760716 +0000 UTC m=+13.897486321"
Oct 30 21:33:20.283315 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:20.283272    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xrkmw" event={"ID":"32dd7eaa-b8a6-4de5-a13a-0640b94801e8","Type":"ContainerStarted","Data":"f94297ef4f87b3ed0a014e57e47bc0ebfc91c7ca28b32a67e6af1fe054fe3a4c"}
Oct 30 21:33:20.285785 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:20.285746    2574 generic.go:356] "Generic (PLEG): container finished" podID="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" containerID="cf6df429750269077117dc72dfd54070b78d4ffc9ab2f4c1b7d3304938f5e763" exitCode=0
Oct 30 21:33:20.285931 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:20.285833    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kx4s8" event={"ID":"3adefe8b-2ffa-4ee1-aad4-a658970e79aa","Type":"ContainerDied","Data":"cf6df429750269077117dc72dfd54070b78d4ffc9ab2f4c1b7d3304938f5e763"}
Oct 30 21:33:20.399742 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:20.399680    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xrkmw" podStartSLOduration=3.56486573 podStartE2EDuration="14.399659124s" totalImagesPullingTime="10.834793394s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:20.398956684 +0000 UTC m=+14.789682287" watchObservedRunningTime="2026-10-30 21:33:20.399659124 +0000 UTC m=+14.790384738"
Oct 30 21:33:21.165166 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:21.165129    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:21.165374 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:21.165186    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:21.165374 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:21.165296    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:21.165374 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:21.165314    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:21.165529 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:21.165389    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:21.165529 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:21.165464    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:21.870945 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:21.870856    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:22.044437 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:22.044397    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:22.221220 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:22.221125    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:22.292685 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:22.292648    2574 generic.go:356] "Generic (PLEG): container finished" podID="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" containerID="2991e61a11e7fb3fa3ac422f788e510f32ba7dca81f64e65ee479126df5672ec" exitCode=0
Oct 30 21:33:22.292847 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:22.292706    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kx4s8" event={"ID":"3adefe8b-2ffa-4ee1-aad4-a658970e79aa","Type":"ContainerDied","Data":"2991e61a11e7fb3fa3ac422f788e510f32ba7dca81f64e65ee479126df5672ec"}
Oct 30 21:33:22.385159 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:22.385122    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:23.002910 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:23.002872    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:23.003357 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:23.002941    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:23.003357 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:23.003057    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:23.003357 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:23.003138    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret podName:fde4a00d-016e-4670-a530-f61d1e193a9c nodeName:}" failed. No retries permitted until 2026-10-30 21:33:39.003116435 +0000 UTC m=+33.393842039 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret") pod "global-pull-secret-syncer-ztncc" (UID: "fde4a00d-016e-4670-a530-f61d1e193a9c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:23.003357 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:23.003057    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:23.003357 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:23.003247    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs podName:6ad46264-9367-4a4e-8eb9-eb0d0470a669 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:39.003211524 +0000 UTC m=+33.393937122 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs") pod "network-metrics-daemon-w9rmz" (UID: "6ad46264-9367-4a4e-8eb9-eb0d0470a669") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:23.103898 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:23.103856    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx4xs\" (UniqueName: \"kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs\") pod \"network-check-target-2r74t\" (UID: \"5c24d279-0f41-4c11-b197-c7c9ce7ed028\") " pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:23.104061 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:23.104043    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:33:23.104103 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:23.104068    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:33:23.104103 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:23.104083    2574 projected.go:196] Error preparing data for projected volume kube-api-access-fx4xs for pod openshift-network-diagnostics/network-check-target-2r74t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:23.104189 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:23.104139    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs podName:5c24d279-0f41-4c11-b197-c7c9ce7ed028 nodeName:}" failed. No retries permitted until 2026-10-30 21:33:39.104122001 +0000 UTC m=+33.494847587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fx4xs" (UniqueName: "kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs") pod "network-check-target-2r74t" (UID: "5c24d279-0f41-4c11-b197-c7c9ce7ed028") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:23.165419 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:23.165381    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:23.165419 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:23.165418    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:23.165635 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:23.165381    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:23.165635 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:23.165508    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:23.165635 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:23.165583    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:23.165744 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:23.165652    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:23.354840 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:23.354605    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:23.518994 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:23.518961    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:23.923670 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:23.923425    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:24.300788 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:24.300757    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs" event={"ID":"c5deaa23-062e-431f-bca9-ffb4b60159cd","Type":"ContainerStarted","Data":"10d93ef757e4084d5243ccdd09a54dbd5deb91afb434688a4a76ce699704612d"}
Oct 30 21:33:24.301217 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:24.300829    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs" event={"ID":"c5deaa23-062e-431f-bca9-ffb4b60159cd","Type":"ContainerStarted","Data":"21a4784a8ca36dbaed879f8d67785111aceec74db90c127475ebe23a2d298e0a"}
Oct 30 21:33:24.301217 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:24.300858    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs" event={"ID":"c5deaa23-062e-431f-bca9-ffb4b60159cd","Type":"ContainerStarted","Data":"ff5664f4bcf2f418585a1a26debf04eafa35ce36f3d779844a3d7e2b25700c65"}
Oct 30 21:33:24.301217 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:24.300886    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs" event={"ID":"c5deaa23-062e-431f-bca9-ffb4b60159cd","Type":"ContainerStarted","Data":"cb2aaa175531b475b3df79c91ee3d56afd1916676d28bb1032918592d517c404"}
Oct 30 21:33:24.301217 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:24.300913    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs" event={"ID":"c5deaa23-062e-431f-bca9-ffb4b60159cd","Type":"ContainerStarted","Data":"fcd10b82290a24f06d96336243d68f9bcf413f30b2240d367737fec014255800"}
Oct 30 21:33:24.301217 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:24.300941    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs" event={"ID":"c5deaa23-062e-431f-bca9-ffb4b60159cd","Type":"ContainerStarted","Data":"c26d976d42cab95f84162a1defacc7bd82d6f2fd7ce229255cbc6febd902c26b"}
Oct 30 21:33:25.165519 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:25.165482    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:25.165519 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:25.165512    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:25.165723 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:25.165510    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:25.165723 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:25.165585    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:25.165723 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:25.165677    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:25.165807 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:25.165750    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:25.305728 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:25.305697    2574 generic.go:356] "Generic (PLEG): container finished" podID="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" containerID="92d4c910b1e531353a66006ea8e6aa8903a713206a4202289ea61282f392d3cc" exitCode=0
Oct 30 21:33:25.306180 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:25.305750    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kx4s8" event={"ID":"3adefe8b-2ffa-4ee1-aad4-a658970e79aa","Type":"ContainerDied","Data":"92d4c910b1e531353a66006ea8e6aa8903a713206a4202289ea61282f392d3cc"}
Oct 30 21:33:25.353493 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:25.353435    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:25.498845 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:25.498759    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:25.930379 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:25.930343    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:26.054797 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:26.054772    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z8g8q_faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8/dns-node-resolver/0.log"
Oct 30 21:33:26.081074 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:26.081005    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:26.310984 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:26.310942    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs" event={"ID":"c5deaa23-062e-431f-bca9-ffb4b60159cd","Type":"ContainerStarted","Data":"9e488600d39a7f98f5b7deea5fe502e392a2d32d76be6a9345bc298ce9a8c254"}
Oct 30 21:33:26.313738 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:26.313714    2574 generic.go:356] "Generic (PLEG): container finished" podID="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" containerID="e963196f920a806acf814a2ad782f745eaec5a80f3fc2a75aa3ca92e733a110c" exitCode=0
Oct 30 21:33:26.313836 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:26.313744    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kx4s8" event={"ID":"3adefe8b-2ffa-4ee1-aad4-a658970e79aa","Type":"ContainerDied","Data":"e963196f920a806acf814a2ad782f745eaec5a80f3fc2a75aa3ca92e733a110c"}
Oct 30 21:33:26.582716 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:26.582684    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:26.961805 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:26.961782    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jgphd_06512914-5083-4ad9-a3da-a6b916f53b6e/node-ca/0.log"
Oct 30 21:33:27.165170 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:27.165131    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:27.165336 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:27.165142    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:27.165336 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:27.165261    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:27.165336 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:27.165148    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:27.165454 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:27.165359    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:27.165454 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:27.165397    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:27.183121 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:27.183095    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:27.319681 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:27.319594    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kx4s8" event={"ID":"3adefe8b-2ffa-4ee1-aad4-a658970e79aa","Type":"ContainerStarted","Data":"f4cf14d3fc0d7f7557470fa65479b1d6dcb91655d80e4b5f671245c3efe75dc1"}
Oct 30 21:33:27.393532 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:27.393504    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:27.438808 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:27.438763    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kx4s8" podStartSLOduration=8.726070929 podStartE2EDuration="21.438749008s" totalImagesPullingTime="12.712678079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:27.438323089 +0000 UTC m=+21.829048691" watchObservedRunningTime="2026-10-30 21:33:27.438749008 +0000 UTC m=+21.829474610"
Oct 30 21:33:27.555421 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:27.555387    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:27.788133 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:27.788060    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:27.970800 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:27.970766    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:28.325162 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:28.324976    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs" event={"ID":"c5deaa23-062e-431f-bca9-ffb4b60159cd","Type":"ContainerStarted","Data":"697328951c58708867007faef70e33bde5f5a0c130c8f1cd2229617d4b6dffe0"}
Oct 30 21:33:28.325776 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:28.325497    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:28.325776 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:28.325559    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:28.340868 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:28.340841    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:28.578738 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:28.578657    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:28.595488 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:28.595423    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs" podStartSLOduration=7.6912038769999995 podStartE2EDuration="22.59541108s" totalImagesPullingTime="14.904207203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:28.544941934 +0000 UTC m=+22.935667537" watchObservedRunningTime="2026-10-30 21:33:28.59541108 +0000 UTC m=+22.986136683"
Oct 30 21:33:28.740557 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:28.740518    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:28.888471 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:28.888402    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:29.050048 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:29.050014    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:29.165459 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:29.165371    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:29.165459 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:29.165454    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:29.165661 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:29.165473    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:29.165661 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:29.165578    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:29.165751 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:29.165676    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:29.165816 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:29.165775    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:29.195373 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:29.195330    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:29.327742 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:29.327714    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:29.342695 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:29.342665    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:33:29.367371 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:29.367326    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:29.782135 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:29.782104    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:29.976423 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:29.976391    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:30.538110 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:30.538039    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:30.691936 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:30.691902    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:30.861087 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:30.860998    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:31.165777 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:31.165692    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:31.165777 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:31.165723    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:31.165976 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:31.165805    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:31.165976 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:31.165817    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:31.165976 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:31.165902    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:31.165976 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:31.165968    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:31.176687 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:31.176660    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:31.387142 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:31.387108    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:31.787106 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:31.787072    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:31.977220 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:31.977187    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:32.573018 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:32.572983    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:33.036956 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:33.036869    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:33.165712 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:33.165676    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:33.165891 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:33.165748    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:33.165891 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:33.165834    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:33.165891 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:33.165848    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:33.166036 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:33.165907    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:33.166036 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:33.165991    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:33.186363 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:33.186335    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:33.377662 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:33.377573    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:33.618176 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:33.618145    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:33.977821 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:33.977786    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:34.133970 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:34.133934    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:34.767710 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:34.767678    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:35.165733 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:35.165655    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:35.166117 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:35.165748    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:35.166117 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:35.165778    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:35.166117 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:35.165784    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:35.166117 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:35.165863    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:35.166117 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:35.165917    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:35.175736 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:35.175708    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:35.388155 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:35.388118    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:35.538053 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:35.537966    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:35.733169 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:35.733142    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:35.990689 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:35.990606    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:36.388091 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:36.388020    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:36.590425 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:36.590386    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:37.165577 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:37.165535    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:37.165759 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:37.165584    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:37.165759 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:37.165548    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:37.165759 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:37.165657    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:37.165759 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:37.165716    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:37.165907 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:37.165763    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:37.273519 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:37.273482    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:37.321212 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:37.321180    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:33:37.321409 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:37.321349    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:33:37.321786 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:37.321757    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:33:38.687916 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:38.687881    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:38.834926 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:38.834892    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:38.998593 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:38.998509    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:39.017540 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:39.017500    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:39.017671 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:39.017567    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:39.017671 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:39.017655    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:39.017744 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:39.017702    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs podName:6ad46264-9367-4a4e-8eb9-eb0d0470a669 nodeName:}" failed. No retries permitted until 2026-10-30 21:34:11.017688454 +0000 UTC m=+65.408414035 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs") pod "network-metrics-daemon-w9rmz" (UID: "6ad46264-9367-4a4e-8eb9-eb0d0470a669") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:33:39.017744 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:39.017655    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:39.017819 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:39.017785    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret podName:fde4a00d-016e-4670-a530-f61d1e193a9c nodeName:}" failed. No retries permitted until 2026-10-30 21:34:11.017770083 +0000 UTC m=+65.408495664 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret") pod "global-pull-secret-syncer-ztncc" (UID: "fde4a00d-016e-4670-a530-f61d1e193a9c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:33:39.118185 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:39.118147    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx4xs\" (UniqueName: \"kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs\") pod \"network-check-target-2r74t\" (UID: \"5c24d279-0f41-4c11-b197-c7c9ce7ed028\") " pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:39.118389 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:39.118336    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:33:39.118389 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:39.118355    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:33:39.118389 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:39.118365    2574 projected.go:196] Error preparing data for projected volume kube-api-access-fx4xs for pod openshift-network-diagnostics/network-check-target-2r74t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:39.118480 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:39.118416    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs podName:5c24d279-0f41-4c11-b197-c7c9ce7ed028 nodeName:}" failed. No retries permitted until 2026-10-30 21:34:11.118401302 +0000 UTC m=+65.509126901 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fx4xs" (UniqueName: "kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs") pod "network-check-target-2r74t" (UID: "5c24d279-0f41-4c11-b197-c7c9ce7ed028") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:33:39.165352 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:39.165325    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:39.165352 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:39.165356    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:39.165567 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:39.165429    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:39.165567 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:39.165514    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:39.165630 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:39.165567    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:39.165664 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:39.165626    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:39.722213 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:39.722179    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:39.884391 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:39.884361    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:41.165400 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:41.165360    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:41.165400 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:41.165386    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:41.165923 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:41.165361    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:41.165923 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:41.165487    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:41.165923 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:41.165536    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:41.165923 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:41.165606    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:43.165510 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:43.165474    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:43.166016 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:43.165531    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:43.166016 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:43.165638    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:43.166016 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:43.165658    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:43.166016 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:43.165727    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:43.166016 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:43.165797    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:44.037583 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:44.037548    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:44.186306 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:44.186269    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:45.165732 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:45.165703    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:45.165943 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:45.165793    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:45.165943 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:45.165805    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:45.165943 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:45.165884    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:45.166077 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:45.165943    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:45.166077 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:45.166008    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:45.267695 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:45.267659    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:45.460186 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:45.460110    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:45.852259 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:45.852157    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:46.004221 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:46.004187    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:47.165318 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:47.165282    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:47.165746 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:47.165396    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:47.165746 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:47.165408    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:47.165746 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:47.165283    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:47.165746 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:47.165510    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:47.165746 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:47.165569    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:47.757752 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:47.757513    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2r74t"]
Oct 30 21:33:47.757936 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:47.757855    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:47.758003 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:47.757969    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:47.758831 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:47.758339    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ztncc"]
Oct 30 21:33:47.758831 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:47.758471    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:47.758831 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:47.758636    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:47.759961 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:47.759936    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w9rmz"]
Oct 30 21:33:47.760183 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:47.760164    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:47.760324 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:47.760293    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:49.164958 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:49.164922    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:49.165357 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:49.164944    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:49.165357 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:49.165054    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:49.165357 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:49.165102    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:49.835502 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:49.835467    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:50.011708 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:50.011670    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:50.168091 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:50.168065    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:50.168551 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:50.168163    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:50.181483 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:50.181456    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:50.342563 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:50.342536    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:50.497398 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:50.497320    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:50.799664 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:50.799582    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:50.949946 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:50.949911    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:51.165125 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:51.165103    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:51.165310 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:51.165215    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9rmz" podUID="6ad46264-9367-4a4e-8eb9-eb0d0470a669"
Oct 30 21:33:51.165310 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:51.165271    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:51.165388 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:51.165368    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ztncc" podUID="fde4a00d-016e-4670-a530-f61d1e193a9c"
Oct 30 21:33:52.165199 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.165156    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:52.165719 ip-10-0-130-104 kubenswrapper[2574]: E1030 21:33:52.165333    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r74t" podUID="5c24d279-0f41-4c11-b197-c7c9ce7ed028"
Oct 30 21:33:52.472628 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.472548    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeReady"
Oct 30 21:33:52.472769 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.472674    2574 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 21:33:52.627331 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.627293    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5svzl"]
Oct 30 21:33:52.629004 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.628988    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-csjtg"]
Oct 30 21:33:52.629125 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.629106    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:52.630624 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.630593    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.635452 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.635429    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dskw4"]
Oct 30 21:33:52.638314 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.636895    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dskw4"
Oct 30 21:33:52.683243 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.683212    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:33:52.683610 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.683592    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-26jx5\""
Oct 30 21:33:52.683782 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.683743    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:33:52.683782 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.683748    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:33:52.683782 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.683765    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:33:52.683782 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.683777    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:33:52.683945 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.683787    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pczpt\""
Oct 30 21:33:52.688396 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.688377    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:33:52.688875 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.688861    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:33:52.688953 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.688896    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:33:52.688953 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.688913    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:33:52.731397 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.731323    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-flltm\""
Oct 30 21:33:52.732385 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.732367    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dskw4"]
Oct 30 21:33:52.732881 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.732862    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-csjtg"]
Oct 30 21:33:52.733330 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.733316    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5svzl"]
Oct 30 21:33:52.811063 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.811021    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwxx8\" (UniqueName: \"kubernetes.io/projected/76696315-4c5d-4460-a3ab-e027a13211e3-kube-api-access-rwxx8\") pod \"ingress-canary-dskw4\" (UID: \"76696315-4c5d-4460-a3ab-e027a13211e3\") " pod="openshift-ingress-canary/ingress-canary-dskw4"
Oct 30 21:33:52.811063 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.811058    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bdceec21-dc8d-4813-9637-d9b59174f16f-config-volume\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:52.811314 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.811079    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3bfdbb07-013c-4ab3-bf93-05af731a7b13-crio-socket\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.811314 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.811101    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdceec21-dc8d-4813-9637-d9b59174f16f-tmp-dir\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:52.811314 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.811149    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tl6d\" (UniqueName: \"kubernetes.io/projected/3bfdbb07-013c-4ab3-bf93-05af731a7b13-kube-api-access-4tl6d\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.811314 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.811190    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xk77\" (UniqueName: \"kubernetes.io/projected/bdceec21-dc8d-4813-9637-d9b59174f16f-kube-api-access-4xk77\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:52.811314 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.811252    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76696315-4c5d-4460-a3ab-e027a13211e3-cert\") pod \"ingress-canary-dskw4\" (UID: \"76696315-4c5d-4460-a3ab-e027a13211e3\") " pod="openshift-ingress-canary/ingress-canary-dskw4"
Oct 30 21:33:52.811314 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.811282    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3bfdbb07-013c-4ab3-bf93-05af731a7b13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.811314 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.811304    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdceec21-dc8d-4813-9637-d9b59174f16f-metrics-tls\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:52.811547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.811319    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bfdbb07-013c-4ab3-bf93-05af731a7b13-data-volume\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.811547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.811437    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3bfdbb07-013c-4ab3-bf93-05af731a7b13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.904188 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.904161    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:52.911866 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.911834    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwxx8\" (UniqueName: \"kubernetes.io/projected/76696315-4c5d-4460-a3ab-e027a13211e3-kube-api-access-rwxx8\") pod \"ingress-canary-dskw4\" (UID: \"76696315-4c5d-4460-a3ab-e027a13211e3\") " pod="openshift-ingress-canary/ingress-canary-dskw4"
Oct 30 21:33:52.911866 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.911868    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bdceec21-dc8d-4813-9637-d9b59174f16f-config-volume\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:52.912758 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.911885    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3bfdbb07-013c-4ab3-bf93-05af731a7b13-crio-socket\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.912758 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.911903    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdceec21-dc8d-4813-9637-d9b59174f16f-tmp-dir\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:52.912758 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.911954    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tl6d\" (UniqueName: \"kubernetes.io/projected/3bfdbb07-013c-4ab3-bf93-05af731a7b13-kube-api-access-4tl6d\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.912758 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.912009    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xk77\" (UniqueName: \"kubernetes.io/projected/bdceec21-dc8d-4813-9637-d9b59174f16f-kube-api-access-4xk77\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:52.912758 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.912046    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76696315-4c5d-4460-a3ab-e027a13211e3-cert\") pod \"ingress-canary-dskw4\" (UID: \"76696315-4c5d-4460-a3ab-e027a13211e3\") " pod="openshift-ingress-canary/ingress-canary-dskw4"
Oct 30 21:33:52.912758 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.912080    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3bfdbb07-013c-4ab3-bf93-05af731a7b13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.912758 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.912103    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3bfdbb07-013c-4ab3-bf93-05af731a7b13-crio-socket\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.912758 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.912115    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdceec21-dc8d-4813-9637-d9b59174f16f-metrics-tls\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:52.912758 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.912189    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bfdbb07-013c-4ab3-bf93-05af731a7b13-data-volume\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.912758 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.912275    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3bfdbb07-013c-4ab3-bf93-05af731a7b13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.912758 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.912334    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdceec21-dc8d-4813-9637-d9b59174f16f-tmp-dir\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:52.912758 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.912618    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bfdbb07-013c-4ab3-bf93-05af731a7b13-data-volume\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.913414 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.912905    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3bfdbb07-013c-4ab3-bf93-05af731a7b13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.913414 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.913057    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bdceec21-dc8d-4813-9637-d9b59174f16f-config-volume\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:52.916084 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.916061    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdceec21-dc8d-4813-9637-d9b59174f16f-metrics-tls\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:52.916196 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.916094    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3bfdbb07-013c-4ab3-bf93-05af731a7b13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.916196 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.916142    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76696315-4c5d-4460-a3ab-e027a13211e3-cert\") pod \"ingress-canary-dskw4\" (UID: \"76696315-4c5d-4460-a3ab-e027a13211e3\") " pod="openshift-ingress-canary/ingress-canary-dskw4"
Oct 30 21:33:52.971586 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.971554    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tl6d\" (UniqueName: \"kubernetes.io/projected/3bfdbb07-013c-4ab3-bf93-05af731a7b13-kube-api-access-4tl6d\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:52.974529 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.974495    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwxx8\" (UniqueName: \"kubernetes.io/projected/76696315-4c5d-4460-a3ab-e027a13211e3-kube-api-access-rwxx8\") pod \"ingress-canary-dskw4\" (UID: \"76696315-4c5d-4460-a3ab-e027a13211e3\") " pod="openshift-ingress-canary/ingress-canary-dskw4"
Oct 30 21:33:52.974529 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:52.974519    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xk77\" (UniqueName: \"kubernetes.io/projected/bdceec21-dc8d-4813-9637-d9b59174f16f-kube-api-access-4xk77\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:53.055294 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:53.055206    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:53.165069 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:53.165036    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:33:53.165069 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:53.165077    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:33:53.217551 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:53.217509    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:33:53.217551 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:53.217509    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-qlt7z\""
Oct 30 21:33:53.217551 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:53.217558    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-76thg\""
Oct 30 21:33:53.218055 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:53.217635    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:33:53.244807 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:53.244772    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:53.250757 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:53.250732    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:33:53.254527 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:53.254503    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dskw4"
Oct 30 21:33:53.738319 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:53.738284    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dskw4"]
Oct 30 21:33:53.740086 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:53.740065    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5svzl"]
Oct 30 21:33:53.745856 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:53.745192    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76696315_4c5d_4460_a3ab_e027a13211e3.slice/crio-e603218bcec4ff474ecda24159dea659b950347d7ea0886ebe06d5c16cfa3f2c WatchSource:0}: Error finding container e603218bcec4ff474ecda24159dea659b950347d7ea0886ebe06d5c16cfa3f2c: Status 404 returned error can't find the container with id e603218bcec4ff474ecda24159dea659b950347d7ea0886ebe06d5c16cfa3f2c
Oct 30 21:33:53.747107 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:53.746994    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-csjtg"]
Oct 30 21:33:53.790563 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:53.790536    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bfdbb07_013c_4ab3_bf93_05af731a7b13.slice/crio-917d65d0613fff8e31449f6ec9d568db040849a69afd20001b7f89b436eb3be3 WatchSource:0}: Error finding container 917d65d0613fff8e31449f6ec9d568db040849a69afd20001b7f89b436eb3be3: Status 404 returned error can't find the container with id 917d65d0613fff8e31449f6ec9d568db040849a69afd20001b7f89b436eb3be3
Oct 30 21:33:54.165287 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:54.165225    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:33:54.217544 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:54.217484    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:33:54.217544 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:54.217533    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-cprz9\""
Oct 30 21:33:54.217981 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:54.217605    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:33:54.386743 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:54.386699    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dskw4" event={"ID":"76696315-4c5d-4460-a3ab-e027a13211e3","Type":"ContainerStarted","Data":"e603218bcec4ff474ecda24159dea659b950347d7ea0886ebe06d5c16cfa3f2c"}
Oct 30 21:33:54.388128 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:54.388099    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-csjtg" event={"ID":"3bfdbb07-013c-4ab3-bf93-05af731a7b13","Type":"ContainerStarted","Data":"9921f2a3224b7bf82c4338ace7e6e12d651343db859b9f01eae29138f1a9ec31"}
Oct 30 21:33:54.388262 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:54.388151    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-csjtg" event={"ID":"3bfdbb07-013c-4ab3-bf93-05af731a7b13","Type":"ContainerStarted","Data":"917d65d0613fff8e31449f6ec9d568db040849a69afd20001b7f89b436eb3be3"}
Oct 30 21:33:54.389113 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:54.389092    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5svzl" event={"ID":"bdceec21-dc8d-4813-9637-d9b59174f16f","Type":"ContainerStarted","Data":"6094eb68105248fad3a349c792a3b857fba7a252a85c8af4d13c346417168a4a"}
Oct 30 21:33:54.496552 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:54.496468    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:54.636749 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:54.636716    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:55.395784 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:55.395691    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-csjtg" event={"ID":"3bfdbb07-013c-4ab3-bf93-05af731a7b13","Type":"ContainerStarted","Data":"3cb62b8ef0a2cbef2405fb7b0d9bfd29e10491c92580f507f91f7f80fdbcf548"}
Oct 30 21:33:55.857265 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:55.857184    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:56.214513 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.214480    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:56.384851 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.384768    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:56.402045 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.402011    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5svzl" event={"ID":"bdceec21-dc8d-4813-9637-d9b59174f16f","Type":"ContainerStarted","Data":"d460df57a14dcdfac7edc904733b57cfc17fdb2f8122eceaa758b218a414e13a"}
Oct 30 21:33:56.402556 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.402070    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5svzl" event={"ID":"bdceec21-dc8d-4813-9637-d9b59174f16f","Type":"ContainerStarted","Data":"56e1b4c5f4e5416f4a91ab8ddcd1e303a9db11ade65646b48fc9195a13587164"}
Oct 30 21:33:56.402556 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.402383    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5svzl"
Oct 30 21:33:56.517874 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.517807    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-5svzl" podStartSLOduration=2.21187301 podStartE2EDuration="4.517786796s" totalImagesPullingTime="2.305913786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:56.516204053 +0000 UTC m=+50.906929715" watchObservedRunningTime="2026-10-30 21:33:56.517786796 +0000 UTC m=+50.908512398"
Oct 30 21:33:56.656442 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.656308    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:56.812452 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.812405    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:56.836795 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.835635    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"]
Oct 30 21:33:56.837870 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.837841    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:33:56.890721 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.890594    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:33:56.890721 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.890630    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:33:56.890721 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.890673    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:33:56.890946 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.890756    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-njnn9\""
Oct 30 21:33:56.890946 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.890820    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 21:33:56.940461 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.940393    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:33:56.941300 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.941221    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv56g\" (UniqueName: \"kubernetes.io/projected/b40cbded-e2dc-420d-b454-eec7a903571e-kube-api-access-kv56g\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:33:56.941421 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.941322    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b40cbded-e2dc-420d-b454-eec7a903571e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:33:56.941475 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.941444    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b40cbded-e2dc-420d-b454-eec7a903571e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:33:56.941530 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.941504    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"]
Oct 30 21:33:56.941530 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.941517    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b40cbded-e2dc-420d-b454-eec7a903571e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:33:56.956706 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.956675    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"]
Oct 30 21:33:56.963919 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.963884    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zts42"]
Oct 30 21:33:56.964066 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.964006    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:56.965748 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.965728    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:56.978513 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:56.978486    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:57.019325 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.019295    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:33:57.019325 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.019316    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-xxswp\""
Oct 30 21:33:57.019486 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.019359    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:33:57.019486 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.019473    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 21:33:57.019705 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.019520    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 21:33:57.019705 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.019592    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l2n6f\""
Oct 30 21:33:57.020611 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.020578    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"]
Oct 30 21:33:57.042227 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.042206    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:33:57.042389 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.042367    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b40cbded-e2dc-420d-b454-eec7a903571e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:33:57.042440 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.042426    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv56g\" (UniqueName: \"kubernetes.io/projected/b40cbded-e2dc-420d-b454-eec7a903571e-kube-api-access-kv56g\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:33:57.042477 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.042454    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b40cbded-e2dc-420d-b454-eec7a903571e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:33:57.042513 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.042478    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b40cbded-e2dc-420d-b454-eec7a903571e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:33:57.043108 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.043089    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b40cbded-e2dc-420d-b454-eec7a903571e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:33:57.044603 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.044578    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b40cbded-e2dc-420d-b454-eec7a903571e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:33:57.044786 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.044771    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b40cbded-e2dc-420d-b454-eec7a903571e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:33:57.103426 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.103390    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv56g\" (UniqueName: \"kubernetes.io/projected/b40cbded-e2dc-420d-b454-eec7a903571e-kube-api-access-kv56g\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:33:57.118666 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.118628    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:33:57.131329 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.131301    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:57.143096 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143064    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-tls\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.143293 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143119    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjsjw\" (UniqueName: \"kubernetes.io/projected/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-api-access-qjsjw\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.143293 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143201    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-sys\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.143293 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143278    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-wtmp\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.143452 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143353    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-root\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.143452 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143387    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgfc5\" (UniqueName: \"kubernetes.io/projected/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-kube-api-access-tgfc5\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.143452 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143442    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-textfile\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.143593 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143508    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.143593 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143539    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.143593 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143569    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-metrics-client-ca\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.143709 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143625    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.143709 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143665    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-accelerators-collector-config\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.143781 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143708    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3ed2089-3110-405e-928e-dcdf13e4d4e3-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.143814 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143780    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.143843 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.143811    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c3ed2089-3110-405e-928e-dcdf13e4d4e3-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.153175 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.153148    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:33:57.247069 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247029    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-textfile\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.247226 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247084    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.247226 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247112    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.247226 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247140    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-metrics-client-ca\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.247226 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247170    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.247226 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247195    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-accelerators-collector-config\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.247226 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247223    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3ed2089-3110-405e-928e-dcdf13e4d4e3-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.247567 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247297    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.247567 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247322    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c3ed2089-3110-405e-928e-dcdf13e4d4e3-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.247567 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247349    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-tls\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.247567 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247387    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjsjw\" (UniqueName: \"kubernetes.io/projected/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-api-access-qjsjw\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.247567 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247413    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-sys\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.247567 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247436    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-wtmp\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.247567 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247459    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-root\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.247567 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247483    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgfc5\" (UniqueName: \"kubernetes.io/projected/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-kube-api-access-tgfc5\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.247984 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.247959    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.248160 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.248139    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-sys\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.248360 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.248338    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c3ed2089-3110-405e-928e-dcdf13e4d4e3-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.248431 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.248357    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-wtmp\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.248479 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.248437    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-root\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.248588 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.248561    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-textfile\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.248847 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.248819    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-accelerators-collector-config\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.249022 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.249000    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-metrics-client-ca\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.249079 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.249054    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3ed2089-3110-405e-928e-dcdf13e4d4e3-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.250677 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.250630    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.250826 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.250808    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-tls\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.250911 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.250889    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.251481 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.251457    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.278776 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.278743    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:57.304284 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.304256    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgfc5\" (UniqueName: \"kubernetes.io/projected/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-kube-api-access-tgfc5\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.304411 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.304398    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjsjw\" (UniqueName: \"kubernetes.io/projected/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-api-access-qjsjw\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.406189 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.406148    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dskw4" event={"ID":"76696315-4c5d-4460-a3ab-e027a13211e3","Type":"ContainerStarted","Data":"6ac881de210bf28bca7a197745e03a498c4b05542b82940e222492f2e69bed00"}
Oct 30 21:33:57.408324 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.408299    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-csjtg" event={"ID":"3bfdbb07-013c-4ab3-bf93-05af731a7b13","Type":"ContainerStarted","Data":"891ac68af2e800c2da5b1da812eceb03285ec593b1c870e06115255fd0c488a8"}
Oct 30 21:33:57.481974 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.481888    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:57.542391 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.542333    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dskw4" podStartSLOduration=2.310153816 podStartE2EDuration="5.542318941s" totalImagesPullingTime="3.232165125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:57.54164012 +0000 UTC m=+51.932365742" watchObservedRunningTime="2026-10-30 21:33:57.542318941 +0000 UTC m=+51.933044544"
Oct 30 21:33:57.577442 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.577411    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:33:57.583080 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.583046    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:33:57.594995 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:57.594950    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42caab53_09ce_4c8f_b7ab_5d86ae4b64ba.slice/crio-cd58dd4d4967280125b3a50c203b0f98c9312317c5689dab770ad8a25978a729 WatchSource:0}: Error finding container cd58dd4d4967280125b3a50c203b0f98c9312317c5689dab770ad8a25978a729: Status 404 returned error can't find the container with id cd58dd4d4967280125b3a50c203b0f98c9312317c5689dab770ad8a25978a729
Oct 30 21:33:57.609954 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.609921    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"]
Oct 30 21:33:57.615972 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:57.615942    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40cbded_e2dc_420d_b454_eec7a903571e.slice/crio-ef43d67f849f8990c0f276df26e38d8460b0390064ed1f7d8aca27a66e157067 WatchSource:0}: Error finding container ef43d67f849f8990c0f276df26e38d8460b0390064ed1f7d8aca27a66e157067: Status 404 returned error can't find the container with id ef43d67f849f8990c0f276df26e38d8460b0390064ed1f7d8aca27a66e157067
Oct 30 21:33:57.645793 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.645764    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:57.674709 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.674649    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-csjtg" podStartSLOduration=2.641546167 podStartE2EDuration="5.674635648s" totalImagesPullingTime="3.033089481s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:57.673837566 +0000 UTC m=+52.064563195" watchObservedRunningTime="2026-10-30 21:33:57.674635648 +0000 UTC m=+52.065361252"
Oct 30 21:33:57.807249 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.807209    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:57.958695 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:57.958667    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"]
Oct 30 21:33:57.960961 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:33:57.960935    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ed2089_3110_405e_928e_dcdf13e4d4e3.slice/crio-783fbfeafd1df21e0299009f71895686501914bd807dc64b8236efefae7bb09b WatchSource:0}: Error finding container 783fbfeafd1df21e0299009f71895686501914bd807dc64b8236efefae7bb09b: Status 404 returned error can't find the container with id 783fbfeafd1df21e0299009f71895686501914bd807dc64b8236efefae7bb09b
Oct 30 21:33:58.369419 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:58.369388    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:58.412519 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:58.412488    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc" event={"ID":"c3ed2089-3110-405e-928e-dcdf13e4d4e3","Type":"ContainerStarted","Data":"783fbfeafd1df21e0299009f71895686501914bd807dc64b8236efefae7bb09b"}
Oct 30 21:33:58.413592 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:58.413565    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zts42" event={"ID":"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba","Type":"ContainerStarted","Data":"cd58dd4d4967280125b3a50c203b0f98c9312317c5689dab770ad8a25978a729"}
Oct 30 21:33:58.415388 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:58.415364    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck" event={"ID":"b40cbded-e2dc-420d-b454-eec7a903571e","Type":"ContainerStarted","Data":"2c6aef85268285cf42de05e70f4abb894fac88d8f57fc5a0bef9071fcfa44ba2"}
Oct 30 21:33:58.415494 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:58.415417    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck" event={"ID":"b40cbded-e2dc-420d-b454-eec7a903571e","Type":"ContainerStarted","Data":"8cb83eb5461e4e9dfa63eddc3029bd0e52390cbc9f5f8896dda6fe3d0e514f63"}
Oct 30 21:33:58.415494 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:58.415441    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck" event={"ID":"b40cbded-e2dc-420d-b454-eec7a903571e","Type":"ContainerStarted","Data":"ef43d67f849f8990c0f276df26e38d8460b0390064ed1f7d8aca27a66e157067"}
Oct 30 21:33:58.562855 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:58.562818    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:59.039346 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.039311    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:59.180710 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.180657    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:59.355968 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.355936    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:59.421377 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.421340    2574 generic.go:356] "Generic (PLEG): container finished" podID="42caab53-09ce-4c8f-b7ab-5d86ae4b64ba" containerID="289f1b7dcc64f924612f95ee3cdb443eb7258d0902b4480a51dd0532db2792c7" exitCode=0
Oct 30 21:33:59.421840 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.421430    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zts42" event={"ID":"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba","Type":"ContainerDied","Data":"289f1b7dcc64f924612f95ee3cdb443eb7258d0902b4480a51dd0532db2792c7"}
Oct 30 21:33:59.424546 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.424520    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck" event={"ID":"b40cbded-e2dc-420d-b454-eec7a903571e","Type":"ContainerStarted","Data":"e69464a4e108a9fc583a17bfa22353e30c7927afa6aebb7ea74fe7b7038506ce"}
Oct 30 21:33:59.608101 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.608062    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"]
Oct 30 21:33:59.610894 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.610871    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.654109 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.654065    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck" podStartSLOduration=2.170711392 podStartE2EDuration="3.654036347s" totalImagesPullingTime="1.483324955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:59.653488398 +0000 UTC m=+54.044214001" watchObservedRunningTime="2026-10-30 21:33:59.654036347 +0000 UTC m=+54.044761950"
Oct 30 21:33:59.666600 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.666569    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 21:33:59.666719 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.666635    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 21:33:59.667510 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.667488    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2kj04h3u19ra3\""
Oct 30 21:33:59.667510 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.667498    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 21:33:59.667664 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.667517    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-th4t8\""
Oct 30 21:33:59.667664 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.667623    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 21:33:59.671677 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.671656    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 21:33:59.672148 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.672125    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.672283 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.672174    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-metrics-client-ca\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.672355 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.672307    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.672406 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.672389    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4jlz\" (UniqueName: \"kubernetes.io/projected/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-kube-api-access-j4jlz\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.672447 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.672432    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-tmp\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.672555 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.672527    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-grpc-tls\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.672666 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.672598    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.672666 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.672645    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-tls\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.672773 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.672702    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.673710 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.673689    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"]
Oct 30 21:33:59.773210 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.773185    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4jlz\" (UniqueName: \"kubernetes.io/projected/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-kube-api-access-j4jlz\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.773361 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.773220    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-tmp\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.773361 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.773289    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-grpc-tls\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.773361 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.773321    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.773361 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.773358    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-tls\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.773505 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.773402    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.773505 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.773454    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.773603 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.773507    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-metrics-client-ca\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.773603 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.773548    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.774086 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.774059    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-tmp\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.775392 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.775316    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-metrics-client-ca\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.776853 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.776824    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.776939 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.776868    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-tls\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.777542 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.777519    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-grpc-tls\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.777676 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.777653    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.777730 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.777698    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.778817 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.778796    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.831038 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.831001    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4jlz\" (UniqueName: \"kubernetes.io/projected/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-kube-api-access-j4jlz\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:33:59.928426 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:33:59.928389    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:00.353000 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:00.352960    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"]
Oct 30 21:34:00.356298 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:34:00.356267    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17bdd0ae_091c_4442_a4d4_4414e8feb7b2.slice/crio-120cf4e69a67840bf831a546a3fdec252d5e015db1b9d335532e6a01e5c77baa WatchSource:0}: Error finding container 120cf4e69a67840bf831a546a3fdec252d5e015db1b9d335532e6a01e5c77baa: Status 404 returned error can't find the container with id 120cf4e69a67840bf831a546a3fdec252d5e015db1b9d335532e6a01e5c77baa
Oct 30 21:34:00.370467 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:00.370432    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:00.431373 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:00.431337    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s" event={"ID":"17bdd0ae-091c-4442-a4d4-4414e8feb7b2","Type":"ContainerStarted","Data":"120cf4e69a67840bf831a546a3fdec252d5e015db1b9d335532e6a01e5c77baa"}
Oct 30 21:34:00.433448 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:00.433417    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc" event={"ID":"c3ed2089-3110-405e-928e-dcdf13e4d4e3","Type":"ContainerStarted","Data":"53d8681f16354757901f285a64d5a82941d9525e2d16f7497a5e3698243ecb8e"}
Oct 30 21:34:00.433676 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:00.433485    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc" event={"ID":"c3ed2089-3110-405e-928e-dcdf13e4d4e3","Type":"ContainerStarted","Data":"63e58c0a4ad2500479aa23b7e640701ddb72c7192f5c7935aea3ad95558e8f45"}
Oct 30 21:34:00.433676 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:00.433514    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc" event={"ID":"c3ed2089-3110-405e-928e-dcdf13e4d4e3","Type":"ContainerStarted","Data":"167e89c5052addafe712920c3e97df5d921a6f689c9d7a6e9ec58cddd3867641"}
Oct 30 21:34:00.435578 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:00.435552    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zts42" event={"ID":"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba","Type":"ContainerStarted","Data":"6b6b6c8fb884cefdf2e56fbc846830348e36614fcd70fcad5e4a10e416508be2"}
Oct 30 21:34:00.435685 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:00.435602    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zts42" event={"ID":"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba","Type":"ContainerStarted","Data":"10d93e1ad829b943c7c90c59833401cf6d5d40b179c6b081b80676db954e02dd"}
Oct 30 21:34:00.522298 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:00.522253    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:00.669732 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:00.669673    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc" podStartSLOduration=2.980059426 podStartE2EDuration="4.669655777s" totalImagesPullingTime="1.689596351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:00.548960595 +0000 UTC m=+54.939686222" watchObservedRunningTime="2026-10-30 21:34:00.669655777 +0000 UTC m=+55.060381379"
Oct 30 21:34:00.669904 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:00.669821    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zts42" podStartSLOduration=3.247141908 podStartE2EDuration="4.66981513s" totalImagesPullingTime="1.339372134s" totalInitContainerRuntime="83.301088ms" isStatefulPod=true podCreationTimestamp="2026-10-30 21:33:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:00.668282971 +0000 UTC m=+55.059008574" watchObservedRunningTime="2026-10-30 21:34:00.66981513 +0000 UTC m=+55.060540733"
Oct 30 21:34:01.023423 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.023341    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:01.170037 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.170005    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:01.345340 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.345261    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:01.371356 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.371321    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:01.529613 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.529573    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:01.657700 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.657666    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"]
Oct 30 21:34:01.661186 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.661163    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:01.683905 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.683867    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:01.691192 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.691159    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvjjm\" (UniqueName: \"kubernetes.io/projected/7f0cf155-c021-465e-a08b-eb2e2aa0935e-kube-api-access-mvjjm\") pod \"monitoring-plugin-77f4f76594-btfmp\" (UID: \"7f0cf155-c021-465e-a08b-eb2e2aa0935e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:01.691357 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.691249    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7f0cf155-c021-465e-a08b-eb2e2aa0935e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-btfmp\" (UID: \"7f0cf155-c021-465e-a08b-eb2e2aa0935e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:01.725373 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.725319    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 21:34:01.725544 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.725377    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-bxch5\""
Oct 30 21:34:01.726679 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.726646    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"]
Oct 30 21:34:01.791898 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.791855    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvjjm\" (UniqueName: \"kubernetes.io/projected/7f0cf155-c021-465e-a08b-eb2e2aa0935e-kube-api-access-mvjjm\") pod \"monitoring-plugin-77f4f76594-btfmp\" (UID: \"7f0cf155-c021-465e-a08b-eb2e2aa0935e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:01.792072 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.791942    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7f0cf155-c021-465e-a08b-eb2e2aa0935e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-btfmp\" (UID: \"7f0cf155-c021-465e-a08b-eb2e2aa0935e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:01.794634 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.794600    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7f0cf155-c021-465e-a08b-eb2e2aa0935e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-btfmp\" (UID: \"7f0cf155-c021-465e-a08b-eb2e2aa0935e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:01.839547 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.839515    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:01.850459 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.850428    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvjjm\" (UniqueName: \"kubernetes.io/projected/7f0cf155-c021-465e-a08b-eb2e2aa0935e-kube-api-access-mvjjm\") pod \"monitoring-plugin-77f4f76594-btfmp\" (UID: \"7f0cf155-c021-465e-a08b-eb2e2aa0935e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:01.982361 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:01.982276    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:02.362556 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:02.362481    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"]
Oct 30 21:34:02.365301 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:34:02.365267    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f0cf155_c021_465e_a08b_eb2e2aa0935e.slice/crio-cd3a25a7a99796cf531c10ebf0bb1c6ed0956feaf3ba31abe2291a30528a721d WatchSource:0}: Error finding container cd3a25a7a99796cf531c10ebf0bb1c6ed0956feaf3ba31abe2291a30528a721d: Status 404 returned error can't find the container with id cd3a25a7a99796cf531c10ebf0bb1c6ed0956feaf3ba31abe2291a30528a721d
Oct 30 21:34:02.443756 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:02.443710    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp" event={"ID":"7f0cf155-c021-465e-a08b-eb2e2aa0935e","Type":"ContainerStarted","Data":"cd3a25a7a99796cf531c10ebf0bb1c6ed0956feaf3ba31abe2291a30528a721d"}
Oct 30 21:34:03.450125 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:03.450086    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s" event={"ID":"17bdd0ae-091c-4442-a4d4-4414e8feb7b2","Type":"ContainerStarted","Data":"200900e3768ddcae14c2fe6b00cd41d9d4423f0ff1f27839901e35c280876502"}
Oct 30 21:34:03.450586 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:03.450157    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s" event={"ID":"17bdd0ae-091c-4442-a4d4-4414e8feb7b2","Type":"ContainerStarted","Data":"0c06e576eb727c171160ae99a440be3d53a58b9bccaf6ab960036b9e165e1519"}
Oct 30 21:34:03.450586 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:03.450200    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s" event={"ID":"17bdd0ae-091c-4442-a4d4-4414e8feb7b2","Type":"ContainerStarted","Data":"6773212cc5f0104929270bae575c82657951326f6f739c92501b753b0c1c03b5"}
Oct 30 21:34:04.457776 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:04.457736    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s" event={"ID":"17bdd0ae-091c-4442-a4d4-4414e8feb7b2","Type":"ContainerStarted","Data":"cf3a127ef9fa29235769e041e233c4e0708775e03d05f6901629d37ef86d8871"}
Oct 30 21:34:04.458294 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:04.457820    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s" event={"ID":"17bdd0ae-091c-4442-a4d4-4414e8feb7b2","Type":"ContainerStarted","Data":"154ddf94145a7e13745d1bfa98865269fc0b6208f879505b147675d276ab04a4"}
Oct 30 21:34:04.458294 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:04.457866    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s" event={"ID":"17bdd0ae-091c-4442-a4d4-4414e8feb7b2","Type":"ContainerStarted","Data":"ce60e6e510af500bfa6224e138fbb98dadaf82b2a47d3332d8c5f65de00cc23d"}
Oct 30 21:34:04.458294 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:04.458006    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:04.459533 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:04.459509    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp" event={"ID":"7f0cf155-c021-465e-a08b-eb2e2aa0935e","Type":"ContainerStarted","Data":"7489d591cee57ca9622c821174c31bd14957c98f88c0b7ca22fcfc620648711d"}
Oct 30 21:34:04.459719 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:04.459702    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:04.465587 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:04.465564    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:04.721268 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:04.721196    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s" podStartSLOduration=2.027389771 podStartE2EDuration="5.72118211s" totalImagesPullingTime="3.693792339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:04.674145141 +0000 UTC m=+59.064870762" watchObservedRunningTime="2026-10-30 21:34:04.72118211 +0000 UTC m=+59.111907712"
Oct 30 21:34:04.791545 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:04.791490    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp" podStartSLOduration=2.063485457 podStartE2EDuration="3.791475238s" totalImagesPullingTime="1.727989781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:04.79107654 +0000 UTC m=+59.181802144" watchObservedRunningTime="2026-10-30 21:34:04.791475238 +0000 UTC m=+59.182200840"
Oct 30 21:34:05.573322 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:05.573284    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:05.753971 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:05.753938    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:05.951202 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:05.951172    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:06.017989 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.017955    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:34:06.023072 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.023048    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.067750 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.067704    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:34:06.088054 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.088012    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 21:34:06.130124 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130087    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130124 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130127    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-config-out\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130392 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130147    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130392 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130196    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x4js\" (UniqueName: \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-kube-api-access-2x4js\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130392 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130259    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130392 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130301    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130392 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130331    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130392 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130363    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130668 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130411    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130668 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130532    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130668 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130572    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130668 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130668 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130664    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130837 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130708    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130837 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130753    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130837 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130788    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130976 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130841    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130976 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130929    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-config\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.130976 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.130969    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.131082 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.131021    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-web-config\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.188510 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.188472    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 21:34:06.188510 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.188492    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 21:34:06.188743 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.188570    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 21:34:06.188743 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.188573    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 21:34:06.188743 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.188574    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 21:34:06.188743 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.188609    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 21:34:06.216144 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.216058    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 21:34:06.216348 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.216197    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 21:34:06.216348 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.216269    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 21:34:06.216348 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.216279    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1gtgf37ikplq8\""
Oct 30 21:34:06.216348 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.216277    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tvbqn\""
Oct 30 21:34:06.216348 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.216324    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 21:34:06.216348 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.216349    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 21:34:06.225792 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.225760    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 21:34:06.225792 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.225765    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:34:06.225792 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.225772    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 21:34:06.232076 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.232053    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.232161 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.232087    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.232161 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.232104    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.232308 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.232288    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.232365 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.232346    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.232418 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.232391    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.232470 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.232416    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.232470 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.232444    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.232982 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.232954    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233084 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233042    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-config\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233153 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233088    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233153 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233127    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-web-config\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233271 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233169    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233271 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233201    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-config-out\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233271 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233226    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233423 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x4js\" (UniqueName: \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-kube-api-access-2x4js\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233423 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233302    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233423 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233315    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233423 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233333    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233423 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233378    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233423 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233409    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233687 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233436    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233792 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233772    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233871 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233850    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.233954 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.233929    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.234993 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.234695    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.235517 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.235471    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.236103 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.235617    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.236103 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.235679    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.236103 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.236055    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.236795 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.236765    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.237004 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.236983    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-config-out\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.237191 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.237163    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-config\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.237433 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.237408    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.237529 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.237512    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.237808 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.237788    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.237991 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.237970    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-web-config\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.238913 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.238887    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.239199 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.239180    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.292667 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.292630    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x4js\" (UniqueName: \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-kube-api-access-2x4js\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.350024 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.349978    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:06.410320 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.410294    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5svzl"
Oct 30 21:34:06.831951 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:06.831923    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:34:06.834271 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:34:06.834228    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0534933_54d7_47c4_9302_523383fa1fc1.slice/crio-d74eae5afc9849a57bed32c7be5be8661ec51221d568f49792c77979844689ab WatchSource:0}: Error finding container d74eae5afc9849a57bed32c7be5be8661ec51221d568f49792c77979844689ab: Status 404 returned error can't find the container with id d74eae5afc9849a57bed32c7be5be8661ec51221d568f49792c77979844689ab
Oct 30 21:34:07.192363 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:07.192310    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:07.339047 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:07.339014    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:07.469855 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:07.469761    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerStarted","Data":"d74eae5afc9849a57bed32c7be5be8661ec51221d568f49792c77979844689ab"}
Oct 30 21:34:07.574503 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:07.574467    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:07.725263 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:07.725159    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:08.257947 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:08.257914    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:08.407170 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:08.407121    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:08.474690 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:08.474657    2574 generic.go:356] "Generic (PLEG): container finished" podID="d0534933-54d7-47c4-9302-523383fa1fc1" containerID="76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49" exitCode=0
Oct 30 21:34:08.474854 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:08.474707    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerDied","Data":"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"}
Oct 30 21:34:08.565496 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:08.565417    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:09.166533 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:09.166500    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:09.325325 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:09.325294    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:10.469641 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:10.469613    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:11.075446 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.075407    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:34:11.075613 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.075481    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:34:11.127057 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.127013    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:34:11.127257 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.127116    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:34:11.138753 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.138722    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:34:11.138898 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.138807    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:34:11.176437 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.176396    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx4xs\" (UniqueName: \"kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs\") pod \"network-check-target-2r74t\" (UID: \"5c24d279-0f41-4c11-b197-c7c9ce7ed028\") " pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:34:11.227648 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.227614    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:34:11.228471 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.228437    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-76thg\""
Oct 30 21:34:11.231251 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.231213    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-qlt7z\""
Oct 30 21:34:11.237514 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.237492    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:34:11.240732 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.240706    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:34:11.289050 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.289000    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:34:11.301685 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.301367    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx4xs\" (UniqueName: \"kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs\") pod \"network-check-target-2r74t\" (UID: \"5c24d279-0f41-4c11-b197-c7c9ce7ed028\") " pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:34:11.618452 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.618375    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w9rmz"]
Oct 30 21:34:11.626645 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.626614    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-cprz9\""
Oct 30 21:34:11.635779 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.635746    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:34:11.658532 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:34:11.658431    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad46264_9367_4a4e_8eb9_eb0d0470a669.slice/crio-51baa23e6df870997ec1e1f920801cfa5ab675cc01b830c2dbae35945eca7391 WatchSource:0}: Error finding container 51baa23e6df870997ec1e1f920801cfa5ab675cc01b830c2dbae35945eca7391: Status 404 returned error can't find the container with id 51baa23e6df870997ec1e1f920801cfa5ab675cc01b830c2dbae35945eca7391
Oct 30 21:34:11.666883 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:11.666857    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ztncc"]
Oct 30 21:34:11.670589 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:34:11.670561    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde4a00d_016e_4670_a530_f61d1e193a9c.slice/crio-32ac583bb348ae778276066d8e4c1a593d4649ee6535fe01d295c7dd6fbf6d9e WatchSource:0}: Error finding container 32ac583bb348ae778276066d8e4c1a593d4649ee6535fe01d295c7dd6fbf6d9e: Status 404 returned error can't find the container with id 32ac583bb348ae778276066d8e4c1a593d4649ee6535fe01d295c7dd6fbf6d9e
Oct 30 21:34:12.017177 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:12.017141    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2r74t"]
Oct 30 21:34:12.030600 ip-10-0-130-104 kubenswrapper[2574]: W1030 21:34:12.030566    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c24d279_0f41_4c11_b197_c7c9ce7ed028.slice/crio-b3ca726df6494fcc0c575a3cd48beb39656c8d74fafcfcabc23968c7fb1941cd WatchSource:0}: Error finding container b3ca726df6494fcc0c575a3cd48beb39656c8d74fafcfcabc23968c7fb1941cd: Status 404 returned error can't find the container with id b3ca726df6494fcc0c575a3cd48beb39656c8d74fafcfcabc23968c7fb1941cd
Oct 30 21:34:12.493728 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:12.493633    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w9rmz" event={"ID":"6ad46264-9367-4a4e-8eb9-eb0d0470a669","Type":"ContainerStarted","Data":"51baa23e6df870997ec1e1f920801cfa5ab675cc01b830c2dbae35945eca7391"}
Oct 30 21:34:12.497048 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:12.497020    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerStarted","Data":"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"}
Oct 30 21:34:12.497206 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:12.497081    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerStarted","Data":"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"}
Oct 30 21:34:12.497206 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:12.497109    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerStarted","Data":"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"}
Oct 30 21:34:12.497206 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:12.497133    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerStarted","Data":"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"}
Oct 30 21:34:12.497206 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:12.497162    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerStarted","Data":"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"}
Oct 30 21:34:12.497206 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:12.497185    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerStarted","Data":"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"}
Oct 30 21:34:12.498409 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:12.498385    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2r74t" event={"ID":"5c24d279-0f41-4c11-b197-c7c9ce7ed028","Type":"ContainerStarted","Data":"b3ca726df6494fcc0c575a3cd48beb39656c8d74fafcfcabc23968c7fb1941cd"}
Oct 30 21:34:12.499509 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:12.499484    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ztncc" event={"ID":"fde4a00d-016e-4670-a530-f61d1e193a9c","Type":"ContainerStarted","Data":"32ac583bb348ae778276066d8e4c1a593d4649ee6535fe01d295c7dd6fbf6d9e"}
Oct 30 21:34:12.630226 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:12.630160    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.742680859 podStartE2EDuration="7.630137129s" totalImagesPullingTime="4.871774231s" totalInitContainerRuntime="15.682039ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:12.629083043 +0000 UTC m=+67.019808671" watchObservedRunningTime="2026-10-30 21:34:12.630137129 +0000 UTC m=+67.020862814"
Oct 30 21:34:13.291328 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:13.290959    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:13.484603 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:13.484563    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:13.505494 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:13.505455    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w9rmz" event={"ID":"6ad46264-9367-4a4e-8eb9-eb0d0470a669","Type":"ContainerStarted","Data":"a3ea7469a1bdef7d1f16182febf0cc74a924c4195f347c6c579de475d43802ff"}
Oct 30 21:34:14.038720 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:14.038673    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:14.189654 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:14.189615    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:14.512788 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:14.512744    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w9rmz" event={"ID":"6ad46264-9367-4a4e-8eb9-eb0d0470a669","Type":"ContainerStarted","Data":"2296cbeb001bc46072252d4589185cf1157bfe9fe7e0fa4839e0cca28761f4e7"}
Oct 30 21:34:14.746370 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:14.746311    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:14.916614 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:14.916578    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:15.170940 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:15.170854    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:15.353774 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:15.353743    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:15.639413 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:15.639324    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:15.840206 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:15.840173    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:16.350870 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:16.350824    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:16.523403 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:16.523362    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2r74t" event={"ID":"5c24d279-0f41-4c11-b197-c7c9ce7ed028","Type":"ContainerStarted","Data":"5354287c7d1a16d5777033a33961324822b3846e73c6eb9ce645f00a820cea7a"}
Oct 30 21:34:16.523574 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:16.523468    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:34:16.637310 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:16.636681    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2r74t" podStartSLOduration=66.985764661 podStartE2EDuration="1m10.636660146s" totalImagesPullingTime="3.650895485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:16.636494481 +0000 UTC m=+71.027220089" watchObservedRunningTime="2026-10-30 21:34:16.636660146 +0000 UTC m=+71.027385741"
Oct 30 21:34:16.637874 ip-10-0-130-104 kubenswrapper[2574]: I1030 21:34:16.637825    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w9rmz" podStartSLOduration=68.998948792 podStartE2EDuration="1m10.637806614s" totalImagesPullingTime="1.638857822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:14.626116324 +0000 UTC m=+69.016841926" watchObservedRunningTime="2026-10-30 21:34:16.637806614 +0000 UTC m=+71.028532218"
Oct 30 21:34:17.254463 ip-10-0-130-104 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 21:34:17.272861 ip-10-0-130-104 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 21:34:17.273124 ip-10-0-130-104 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 21:34:17.273195 ip-10-0-130-104 systemd[1]: kubelet.service: Consumed 4.569s CPU time.
Oct 30 21:34:17.282850 ip-10-0-130-104 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:34:17.356805 ip-10-0-130-104 kubenswrapper[5586]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:34:17.356805 ip-10-0-130-104 kubenswrapper[5586]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:34:17.356805 ip-10-0-130-104 kubenswrapper[5586]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:34:17.356805 ip-10-0-130-104 kubenswrapper[5586]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:34:17.358884 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358802    5586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:17.358884 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358883    5586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:17.358884 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358888    5586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358891    5586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358894    5586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358897    5586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358899    5586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358902    5586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358905    5586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358908    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358910    5586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358913    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358916    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358918    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358921    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358923    5586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358928    5586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358932    5586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358935    5586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358938    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358944    5586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358947    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:17.358978 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358950    5586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358953    5586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358956    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358959    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358962    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358965    5586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358968    5586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358971    5586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358973    5586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358976    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358978    5586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358982    5586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358984    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358987    5586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358990    5586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358993    5586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358996    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.358999    5586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359001    5586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359004    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:17.359490 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359007    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359009    5586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359012    5586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359015    5586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359017    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359020    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359022    5586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359025    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359027    5586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359030    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359032    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359035    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359038    5586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359042    5586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359046    5586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359048    5586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359051    5586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359054    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359056    5586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:17.359973 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359059    5586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359062    5586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359065    5586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359067    5586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359070    5586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359072    5586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359076    5586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359080    5586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359083    5586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359086    5586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359090    5586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359093    5586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359096    5586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359099    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359102    5586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359104    5586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359107    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359109    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359112    5586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359115    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:17.360443 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359117    5586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359119    5586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359122    5586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359124    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359127    5586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359129    5586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359132    5586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359135    5586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359138    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359140    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359143    5586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359146    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359148    5586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359151    5586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359154    5586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359157    5586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359159    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359162    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359164    5586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:17.360930 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359167    5586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359171    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359173    5586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359176    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359179    5586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359182    5586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.359993    5586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360002    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360006    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360010    5586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360013    5586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360017    5586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360021    5586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360024    5586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360027    5586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360030    5586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360033    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360035    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360038    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:17.361430 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360042    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360044    5586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360047    5586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360050    5586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360053    5586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360055    5586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360058    5586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360061    5586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360064    5586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360066    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360069    5586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360072    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360075    5586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360078    5586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360081    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360083    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360086    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360089    5586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360092    5586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:17.361895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360094    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360097    5586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360100    5586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360103    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360105    5586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360108    5586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360111    5586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360113    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360116    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360119    5586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360122    5586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360124    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360127    5586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360129    5586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360132    5586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360135    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360138    5586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360140    5586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360143    5586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360145    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:17.362354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360148    5586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360151    5586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360153    5586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360156    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360158    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360161    5586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360163    5586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360166    5586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360169    5586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360171    5586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360174    5586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360177    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360180    5586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360182    5586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360185    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360187    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360190    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360193    5586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360195    5586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360198    5586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:17.362849 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360201    5586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360203    5586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360205    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360209    5586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360212    5586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360214    5586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360217    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360219    5586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360222    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360225    5586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360227    5586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360242    5586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360245    5586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360248    5586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360251    5586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360254    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360256    5586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360259    5586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360262    5586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:17.363402 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360265    5586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360268    5586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360272    5586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360275    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360278    5586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360281    5586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360284    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360287    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360290    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360293    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360295    5586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360298    5586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360301    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360305    5586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360307    5586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360843    5586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360849    5586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360852    5586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360854    5586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360857    5586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:17.363867 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360860    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360863    5586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360866    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360869    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360872    5586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360875    5586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360878    5586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360880    5586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360883    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360886    5586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360889    5586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360893    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360895    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360898    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360901    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360903    5586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360905    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360909    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360913    5586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:17.364379 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360917    5586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360921    5586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360923    5586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360926    5586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360928    5586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360931    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360934    5586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360936    5586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360939    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360942    5586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360944    5586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360947    5586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360950    5586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360953    5586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360955    5586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360958    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360961    5586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360964    5586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360966    5586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360969    5586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:17.364865 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360971    5586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360974    5586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360977    5586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360979    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360982    5586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360985    5586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360987    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360990    5586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360992    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360995    5586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.360998    5586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361003    5586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361007    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361010    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361012    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361015    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361017    5586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361020    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361022    5586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:17.365372 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361025    5586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361027    5586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361030    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361032    5586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361035    5586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361037    5586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361040    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361042    5586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361045    5586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361049    5586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361052    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361054    5586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361057    5586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361060    5586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361062    5586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361065    5586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361068    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361073    5586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361076    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361079    5586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:17.365965 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361081    5586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361084    5586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361086    5586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361089    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361092    5586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361095    5586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361097    5586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361100    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361103    5586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361105    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361108    5586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361110    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361113    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361116    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361120    5586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361122    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361125    5586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361127    5586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361130    5586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361136    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:17.366856 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361139    5586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361141    5586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361144    5586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361264    5586 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361283    5586 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361290    5586 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361296    5586 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361301    5586 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361304    5586 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361310    5586 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361315    5586 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361318    5586 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361321    5586 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361325    5586 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361329    5586 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361332    5586 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361335    5586 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361339    5586 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361343    5586 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361346    5586 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361349    5586 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361353    5586 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361356    5586 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361359    5586 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:34:17.367378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361362    5586 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361366    5586 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361370    5586 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361373    5586 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361376    5586 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361380    5586 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361383    5586 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361388    5586 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361391    5586 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361394    5586 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361397    5586 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361402    5586 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361405    5586 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361409    5586 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361412    5586 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361415    5586 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361418    5586 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361425    5586 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361429    5586 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361432    5586 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361436    5586 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361439    5586 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361444    5586 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361448    5586 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361451    5586 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:34:17.367943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361455    5586 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361458    5586 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361461    5586 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361465    5586 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361468    5586 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361471    5586 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361474    5586 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361477    5586 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361482    5586 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361485    5586 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361488    5586 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361492    5586 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361495    5586 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361498    5586 flags.go:64] FLAG: --help="false"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361503    5586 flags.go:64] FLAG: --hostname-override="ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361506    5586 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361509    5586 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361513    5586 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361516    5586 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361520    5586 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361523    5586 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361526    5586 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361529    5586 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:34:17.368597 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361532    5586 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361535    5586 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361539    5586 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361542    5586 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361545    5586 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361548    5586 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361551    5586 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361554    5586 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361557    5586 flags.go:64] FLAG: --lock-file=""
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361560    5586 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361563    5586 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361566    5586 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361573    5586 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361576    5586 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361579    5586 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361582    5586 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361585    5586 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361589    5586 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361592    5586 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361595    5586 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361599    5586 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361603    5586 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361607    5586 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361612    5586 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361616    5586 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:34:17.369148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361619    5586 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361622    5586 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361625    5586 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361628    5586 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361631    5586 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361638    5586 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361642    5586 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361645    5586 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361649    5586 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361652    5586 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361655    5586 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361658    5586 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361661    5586 flags.go:64] FLAG: --port="10250"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361664    5586 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361667    5586 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-00fb5a58d9fc2e9b1"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361670    5586 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361673    5586 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361676    5586 flags.go:64] FLAG: --register-node="true"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361679    5586 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361683    5586 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361687    5586 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361690    5586 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361693    5586 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361697    5586 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361700    5586 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361704    5586 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:34:17.369764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361707    5586 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361710    5586 flags.go:64] FLAG: --runonce="false"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361713    5586 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361716    5586 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361721    5586 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361724    5586 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361727    5586 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361730    5586 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361733    5586 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361737    5586 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361740    5586 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361743    5586 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361746    5586 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361749    5586 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361752    5586 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361755    5586 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361758    5586 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361763    5586 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361766    5586 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361769    5586 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361773    5586 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361776    5586 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361778    5586 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361782    5586 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361785    5586 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:34:17.370394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361788    5586 flags.go:64] FLAG: --v="2"
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361797    5586 flags.go:64] FLAG: --version="false"
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361805    5586 flags.go:64] FLAG: --vmodule=""
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361809    5586 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.361813    5586 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361976    5586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361980    5586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361983    5586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361986    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361988    5586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361991    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361995    5586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.361997    5586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362000    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362002    5586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362005    5586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362008    5586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362010    5586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362012    5586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362015    5586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:17.371003 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362019    5586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362022    5586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362025    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362028    5586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362031    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362033    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362036    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362038    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362041    5586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362043    5586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362046    5586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362048    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362051    5586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362053    5586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362056    5586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362059    5586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362062    5586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362064    5586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362067    5586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362071    5586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:17.371522 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362074    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362077    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362079    5586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362082    5586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362085    5586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362087    5586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362090    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362092    5586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362095    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362097    5586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362100    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362102    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362105    5586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362108    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362110    5586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362113    5586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362116    5586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362118    5586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362121    5586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:17.371986 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362123    5586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362126    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362129    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362131    5586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362134    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362137    5586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362139    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362142    5586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362144    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362148    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362150    5586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362153    5586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362157    5586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362159    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362161    5586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362164    5586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362166    5586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362169    5586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362172    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362174    5586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:17.372462 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362177    5586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362179    5586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362182    5586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362185    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362188    5586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362190    5586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362193    5586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362195    5586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362198    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362200    5586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362203    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362206    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362208    5586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362211    5586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362213    5586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362216    5586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362218    5586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362221    5586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362223    5586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:17.372984 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362226    5586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362240    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362243    5586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362246    5586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362249    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362255    5586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362259    5586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362262    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362265    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362268    5586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362271    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362273    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362276    5586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.362303    5586 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362339    5586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362344    5586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:17.373511 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362348    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362351    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362354    5586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362356    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362359    5586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362362    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362365    5586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362368    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362370    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362373    5586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362376    5586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362378    5586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362381    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362383    5586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362386    5586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362388    5586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362391    5586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362394    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362397    5586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362399    5586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:17.373895 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362403    5586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362407    5586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362410    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362413    5586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362415    5586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362418    5586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362420    5586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362423    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362426    5586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362429    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362431    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362434    5586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362438    5586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362440    5586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362443    5586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362446    5586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362448    5586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362451    5586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362453    5586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:17.374423 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362456    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362459    5586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362461    5586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362464    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362468    5586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362471    5586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362475    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362478    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362480    5586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362483    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362485    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362488    5586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362491    5586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362493    5586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362497    5586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362500    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362503    5586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362505    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362508    5586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362510    5586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:17.374924 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362513    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362515    5586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362518    5586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362520    5586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362523    5586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362527    5586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362530    5586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362534    5586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362538    5586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362541    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362543    5586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362546    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362549    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362551    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362554    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362557    5586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362559    5586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362562    5586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362565    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:17.375427 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362567    5586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362570    5586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362573    5586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362575    5586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362578    5586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362580    5586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362583    5586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362587    5586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362591    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362594    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362597    5586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362599    5586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362602    5586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362604    5586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362607    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362609    5586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362612    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362615    5586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362626    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:17.375999 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362630    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362632    5586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362636    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362638    5586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362641    5586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362645    5586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.362647    5586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.368221    5586 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.368256    5586 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.368282    5586 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.368287    5586 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368349    5586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368355    5586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368360    5586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368363    5586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368367    5586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368370    5586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368372    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368375    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368378    5586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:17.376725 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368381    5586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368383    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368386    5586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368389    5586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368392    5586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368394    5586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368398    5586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368400    5586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368404    5586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368407    5586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368410    5586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368413    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368415    5586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368418    5586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368421    5586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368424    5586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368426    5586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368429    5586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368433    5586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368436    5586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:17.377283 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368438    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368441    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368444    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368446    5586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368449    5586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368451    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368454    5586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368457    5586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368460    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368463    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368465    5586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368468    5586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368471    5586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368487    5586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368492    5586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368495    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368498    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368510    5586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:17.377762 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368516    5586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368520    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368524    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368527    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368530    5586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368533    5586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368536    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368538    5586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368541    5586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368544    5586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368546    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368549    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368552    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368554    5586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368557    5586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368560    5586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368562    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368565    5586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368567    5586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:17.378199 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368570    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368572    5586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368575    5586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368578    5586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368580    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368583    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368586    5586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368588    5586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368591    5586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368594    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368597    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368599    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368602    5586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368611    5586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368614    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368617    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368619    5586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368622    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368624    5586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368627    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:17.378823 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368630    5586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368632    5586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368635    5586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368637    5586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368640    5586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368643    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368645    5586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368648    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368651    5586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368653    5586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368656    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368658    5586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368661    5586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368663    5586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368666    5586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368669    5586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368671    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368674    5586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368677    5586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:17.379320 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368680    5586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.368708    5586 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368872    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368878    5586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368881    5586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368884    5586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368887    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368890    5586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368893    5586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368896    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368898    5586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368901    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368904    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368906    5586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368909    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368912    5586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:17.379779 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368914    5586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368917    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368921    5586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368925    5586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368928    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368930    5586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368933    5586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368936    5586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368938    5586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368941    5586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368943    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368946    5586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368949    5586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368951    5586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368954    5586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368956    5586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368959    5586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368962    5586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368965    5586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368968    5586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:17.380227 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368970    5586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368972    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368975    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368977    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368981    5586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368984    5586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368986    5586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368989    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368991    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368994    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368996    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.368999    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369001    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369005    5586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369008    5586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369010    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369013    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369016    5586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369019    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:17.380706 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369022    5586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369025    5586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369027    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369030    5586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369032    5586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369035    5586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369038    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369040    5586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369043    5586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369046    5586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369048    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369051    5586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369053    5586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369056    5586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369059    5586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369062    5586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369064    5586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369067    5586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369070    5586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369073    5586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:17.381158 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369075    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369078    5586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369081    5586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369083    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369086    5586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369088    5586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369091    5586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369093    5586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369096    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369099    5586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369101    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369104    5586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369107    5586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369109    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369112    5586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369114    5586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369117    5586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369121    5586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369125    5586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:17.381708 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369128    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369131    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369134    5586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369136    5586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369139    5586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369141    5586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369144    5586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369147    5586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369149    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369152    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369155    5586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369157    5586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369161    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369164    5586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.369188    5586 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369228    5586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:34:17.382165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369249    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369252    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369255    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369258    5586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369261    5586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369264    5586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369267    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369269    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369272    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369274    5586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369277    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369279    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369282    5586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369285    5586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369287    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369290    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369292    5586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369295    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369298    5586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:34:17.382851 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369301    5586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369304    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369306    5586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369309    5586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369312    5586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369314    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369317    5586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369319    5586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369322    5586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369325    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369328    5586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369331    5586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369333    5586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369336    5586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369339    5586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369342    5586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369344    5586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369347    5586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369349    5586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369352    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:34:17.383588 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369355    5586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369357    5586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369360    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369363    5586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369365    5586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369368    5586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369370    5586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369373    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369376    5586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369378    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369382    5586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369384    5586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369387    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369389    5586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369392    5586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369394    5586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369398    5586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369402    5586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369404    5586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:34:17.384076 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369407    5586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369410    5586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369412    5586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369415    5586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369418    5586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369420    5586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369423    5586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369426    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369428    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369431    5586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369433    5586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369436    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369439    5586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369441    5586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369444    5586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369446    5586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369449    5586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369451    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369457    5586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369460    5586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:34:17.384649 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369462    5586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369465    5586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369468    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369470    5586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369473    5586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369476    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369479    5586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369482    5586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369484    5586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369488    5586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369493    5586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369497    5586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369501    5586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369503    5586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369506    5586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369508    5586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369511    5586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369514    5586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369517    5586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:34:17.385119 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369520    5586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369522    5586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369525    5586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369528    5586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369531    5586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369533    5586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369536    5586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:17.369538    5586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.369593    5586 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.370594    5586 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.370667    5586 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.370856    5586 server.go:1014] "Starting client certificate rotation"
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.370993    5586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.371039    5586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:28:06 +0000 UTC" deadline="2028-07-05 04:22:13.879917705 +0000 UTC"
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.371069    5586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14718h47m56.508850251s"
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.371343    5586 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:34:17.385638 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.372420    5586 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:34:17.386028 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.373881    5586 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:34:17.386028 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.375773    5586 log.go:25] "Validated CRI v1 image API"
Oct 30 21:34:17.386028 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.376551    5586 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:34:17.386028 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.378829    5586 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a7bf4f2f-778c-463f-afb8-1559d8ade28e:/dev/nvme0n1p4 ed8203bc-0c01-4572-bc85-c71eccdc6543:/dev/nvme0n1p3]
Oct 30 21:34:17.386156 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.378851    5586 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/120cf4e69a67840bf831a546a3fdec252d5e015db1b9d335532e6a01e5c77baa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/120cf4e69a67840bf831a546a3fdec252d5e015db1b9d335532e6a01e5c77baa/userdata/shm major:0 minor:320 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1414efc0789b873d1e4beb482d7086f82ea661b7b8fe156ee1808d885f1ea5db/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1414efc0789b873d1e4beb482d7086f82ea661b7b8fe156ee1808d885f1ea5db/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/14cda20d61c1b9e007361243c4d97487f5dd4de58cc27565b4c4d816dbe5ab4c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/14cda20d61c1b9e007361243c4d97487f5dd4de58cc27565b4c4d816dbe5ab4c/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/32ac583bb348ae778276066d8e4c1a593d4649ee6535fe01d295c7dd6fbf6d9e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/32ac583bb348ae778276066d8e4c1a593d4649ee6535fe01d295c7dd6fbf6d9e/userdata/shm major:0 minor:386 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/33ff5cbba2f296cb9e6a579230ee4feb6697a7e90788051c60d95896bd6e5a6a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/33ff5cbba2f296cb9e6a579230ee4feb6697a7e90788051c60d95896bd6e5a6a/userdata/shm major:0 minor:83 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/51baa23e6df870997ec1e1f920801cfa5ab675cc01b830c2dbae35945eca7391/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/51baa23e6df870997ec1e1f920801cfa5ab675cc01b830c2dbae35945eca7391/userdata/shm major:0 minor:385 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5cbd08bc248d2f9c204a3db6c4a799d7ddeafe5e70f3f4527425fb73b9223c1c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5cbd08bc248d2f9c204a3db6c4a799d7ddeafe5e70f3f4527425fb73b9223c1c/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6094eb68105248fad3a349c792a3b857fba7a252a85c8af4d13c346417168a4a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6094eb68105248fad3a349c792a3b857fba7a252a85c8af4d13c346417168a4a/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/783fbfeafd1df21e0299009f71895686501914bd807dc64b8236efefae7bb09b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/783fbfeafd1df21e0299009f71895686501914bd807dc64b8236efefae7bb09b/userdata/shm major:0 minor:261 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/917d65d0613fff8e31449f6ec9d568db040849a69afd20001b7f89b436eb3be3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/917d65d0613fff8e31449f6ec9d568db040849a69afd20001b7f89b436eb3be3/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9c2c8b2b422eca5331baa11b3b69649a1681e5ffe0a5df99f9fe2dd8517588df/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9c2c8b2b422eca5331baa11b3b69649a1681e5ffe0a5df99f9fe2dd8517588df/userdata/shm major:0 minor:95 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a7fde4cea4342ec8f9c95988990190640238b5389c9ea67da85d41f8364a95c4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a7fde4cea4342ec8f9c95988990190640238b5389c9ea67da85d41f8364a95c4/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b3ca726df6494fcc0c575a3cd48beb39656c8d74fafcfcabc23968c7fb1941cd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b3ca726df6494fcc0c575a3cd48beb39656c8d74fafcfcabc23968c7fb1941cd/userdata/shm major:0 minor:390 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cd3a25a7a99796cf531c10ebf0bb1c6ed0956feaf3ba31abe2291a30528a721d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cd3a25a7a99796cf531c10ebf0bb1c6ed0956feaf3ba31abe2291a30528a721d/userdata/shm major:0 minor:326 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cd58dd4d4967280125b3a50c203b0f98c9312317c5689dab770ad8a25978a729/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cd58dd4d4967280125b3a50c203b0f98c9312317c5689dab770ad8a25978a729/userdata/shm major:0 minor:263 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ce77db454de78f3ada0813466ade82455414969ab952277c2fc09bbef09e2a49/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ce77db454de78f3ada0813466ade82455414969ab952277c2fc09bbef09e2a49/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d74eae5afc9849a57bed32c7be5be8661ec51221d568f49792c77979844689ab/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d74eae5afc9849a57bed32c7be5be8661ec51221d568f49792c77979844689ab/userdata/shm major:0 minor:379 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d7cad3ecd7cbdf5061c9f000c7754e145c57d802ba7dac726736f93ac1a2f7dd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d7cad3ecd7cbdf5061c9f000c7754e145c57d802ba7dac726736f93ac1a2f7dd/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dcbfb3ad0699a53006ea978e770fc8ccbd4d48fe7df3751603bb15d74fa4de82/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dcbfb3ad0699a53006ea978e770fc8ccbd4d48fe7df3751603bb15d74fa4de82/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/df91d26b206e05676cd3152cc7a2bedf8c077a01298b4d42b3d273d3834ea190/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/df91d26b206e05676cd3152cc7a2bedf8c077a01298b4d42b3d273d3834ea190/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e603218bcec4ff474ecda24159dea659b950347d7ea0886ebe06d5c16cfa3f2c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e603218bcec4ff474ecda24159dea659b950347d7ea0886ebe06d5c16cfa3f2c/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ef43d67f849f8990c0f276df26e38d8460b0390064ed1f7d8aca27a66e157067/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ef43d67f849f8990c0f276df26e38d8460b0390064ed1f7d8aca27a66e157067/userdata/shm major:0 minor:253 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06512914-5083-4ad9-a3da-a6b916f53b6e/volumes/kubernetes.io~projected/kube-api-access-rbc9d:{mountpoint:/var/lib/kubelet/pods/06512914-5083-4ad9-a3da-a6b916f53b6e/volumes/kubernetes.io~projected/kube-api-access-rbc9d major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~projected/kube-api-access-j4jlz:{mountpoint:/var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~projected/kube-api-access-j4jlz major:0 minor:315 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:314 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:316 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:310 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:313 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:311 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:312 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ccf87bf-a091-4f22-8811-02dcf621e556/volumes/kubernetes.io~projected/kube-api-access-h5rb7:{mountpoint:/var/lib/kubelet/pods/2ccf87bf-a091-4f22-8811-02dcf621e556/volumes/kubernetes.io~projected/kube-api-access-h5rb7 major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3142d576-19ca-48d1-b50f-c22945af6ba3/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/3142d576-19ca-48d1-b50f-c22945af6ba3/volumes/kubernetes.io~secret/agent-certs major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32dd7eaa-b8a6-4de5-a13a-0640b94801e8/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/32dd7eaa-b8a6-4de5-a13a-0640b94801e8/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32dd7eaa-b8a6-4de5-a13a-0640b94801e8/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/32dd7eaa-b8a6-4de5-a13a-0640b94801e8/volumes/kubernetes.io~empty-dir/tmp major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32dd7eaa-b8a6-4de5-a13a-0640b94801e8/volumes/kubernetes.io~projected/kube-api-access-p9mvp:{mountpoint:/var/lib/kubelet/pods/32dd7eaa-b8a6-4de5-a13a-0640b94801e8/volumes/kubernetes.io~projected/kube-api-access-p9mvp major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3adefe8b-2ffa-4ee1-aad4-a658970e79aa/volumes/kubernetes.io~projected/kube-api-access-wk97l:{mountpoint:/var/lib/kubelet/pods/3adefe8b-2ffa-4ee1-aad4-a658970e79aa/volumes/kubernetes.io~projected/kube-api-access-wk97l major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3bfdbb07-013c-4ab3-bf93-05af731a7b13/volumes/kubernetes.io~projected/kube-api-access-4tl6d:{mountpoint:/var/lib/kubelet/pods/3bfdbb07-013c-4ab3-bf93-05af731a7b13/volumes/kubernetes.io~projected/kube-api-access-4tl6d major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3bfdbb07-013c-4ab3-bf93-05af731a7b13/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/3bfdbb07-013c-4ab3-bf93-05af731a7b13/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba/volumes/kubernetes.io~projected/kube-api-access-tgfc5:{mountpoint:/var/lib/kubelet/pods/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba/volumes/kubernetes.io~projected/kube-api-access-tgfc5 major:0 minor:259 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:256 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:258 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5c24d279-0f41-4c11-b197-c7c9ce7ed028/volumes/kubernetes.io~projected/kube-api-access-fx4xs:{mountpoint:/var/lib/kubelet/pods/5c24d279-0f41-4c11-b197-c7c9ce7ed028/volumes/kubernetes.io~projected/kube-api-access-fx4xs major:0 minor:389 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6ad46264-9367-4a4e-8eb9-eb0d0470a669/volumes/kubernetes.io~projected/kube-api-access-c246b:{mountpoint:/var/lib/kubelet/pods/6ad46264-9367-4a4e-8eb9-eb0d0470a669/volumes/kubernetes.io~projected/kube-api-access-c246b major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6ad46264-9367-4a4e-8eb9-eb0d0470a669/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/6ad46264-9367-4a4e-8eb9-eb0d0470a669/volumes/kubernetes.io~secret/metrics-certs major:0 minor:383 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76696315-4c5d-4460-a3ab-e027a13211e3/volumes/kubernetes.io~projected/kube-api-access-rwxx8:{mountpoint:/var/lib/kubelet/pods/76696315-4c5d-4460-a3ab-e027a13211e3/volumes/kubernetes.io~projected/kube-api-access-rwxx8 major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76696315-4c5d-4460-a3ab-e027a13211e3/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/76696315-4c5d-4460-a3ab-e027a13211e3/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7f0cf155-c021-465e-a08b-eb2e2aa0935e/volumes/kubernetes.io~projected/kube-api-access-mvjjm:{mountpoint:/var/lib/kubelet/pods/7f0cf155-c021-465e-a08b-eb2e2aa0935e/volumes/kubernetes.io~projected/kube-api-access-mvjjm major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7f0cf155-c021-465e-a08b-eb2e2aa0935e/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/7f0cf155-c021-465e-a08b-eb2e2aa0935e/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:324 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/90f49eda-5385-490c-9760-2cf569eabccf/volumes/kubernetes.io~projected/kube-api-access-4h8w4:{mountpoint:/var/lib/kubelet/pods/90f49eda-5385-490c-9760-2cf569eabccf/volumes/kubernetes.io~projected/kube-api-access-4h8w4 major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b40cbded-e2dc-420d-b454-eec7a903571e/volumes/kubernetes.io~projected/kube-api-access-kv56g:{mountpoint:/var/lib/kubelet/pods/b40cbded-e2dc-420d-b454-eec7a903571e/volumes/kubernetes.io~projected/kube-api-access-kv56g major:0 minor:252 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b40cbded-e2dc-420d-b454-eec7a903571e/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/b40cbded-e2dc-420d-b454-eec7a903571e/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:243 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b40cbded-e2dc-420d-b454-eec7a903571e/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/b40cbded-e2dc-420d-b454-eec7a903571e/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:244 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bdceec21-dc8d-4813-9637-d9b59174f16f/volumes/kubernetes.io~projected/kube-api-access-4xk77:{mountpoint:/var/lib/kubelet/pods/bdceec21-dc8d-4813-9637-d9b59174f16f/volumes/kubernetes.io~projected/kube-api-access-4xk77 major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bdceec21-dc8d-4813-9637-d9b59174f16f/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/bdceec21-dc8d-4813-9637-d9b59174f16f/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c3ed2089-3110-405e-928e-dcdf13e4d4e3/volumes/kubernetes.io~projected/kube-api-access-qjsjw:{mountpoint:/var/lib/kubelet/pods/c3ed2089-3110-405e-928e-dcdf13e4d4e3/volumes/kubernetes.io~projected/kube-api-access-qjsjw major:0 minor:260 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c3ed2089-3110-405e-928e-dcdf13e4d4e3/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/c3ed2089-3110-405e-928e-dcdf13e4d4e3/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:257 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c3ed2089-3110-405e-928e-dcdf13e4d4e3/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/c3ed2089-3110-405e-928e-dcdf13e4d4e3/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:255 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c5deaa23-062e-431f-bca9-ffb4b60159cd/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/c5deaa23-062e-431f-bca9-ffb4b60159cd/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c5deaa23-062e-431f-bca9-ffb4b60159cd/volumes/kubernetes.io~projected/kube-api-access-kmwdl:{mountpoint:/var/lib/kubelet/pods/c5deaa23-062e-431f-bca9-ffb4b60159cd/volumes/kubernetes.io~projected/kube-api-access-kmwdl major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c5deaa23-062e-431f-bca9-ffb4b60159cd/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/c5deaa23-062e-431f-bca9-ffb4b60159cd/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volume-subpaths/web-config/config-reloader/2 major:0 minor:376 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:376 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volume-subpaths/web-config/prometheus/15 major:0 minor:376 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~empty-dir/config-out major:0 minor:371 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~projected/kube-api-access-2x4js:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~projected/kube-api-access-2x4js major:0 minor:378 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~projected/tls-assets major:0 minor:375 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/config major:0 minor:373 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:370 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:369 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:372 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:377 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:364 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:374 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/web-config major:0 minor:376 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e83d89de-6a2a-4950-a82a-d0705739ce81/volumes/kubernetes.io~projected/kube-api-access-9vplh:{mountpoint:/var/lib/kubelet/pods/e83d89de-6a2a-4950-a82a-d0705739ce81/volumes/kubernetes.io~projected/kube-api-access-9vplh major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8/volumes/kubernetes.io~projected/kube-api-access-s46xr:{mountpoint:/var/lib/kubelet/pods/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8/volumes/kubernetes.io~projected/kube-api-access-s46xr major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fde4a00d-016e-4670-a530-f61d1e193a9c/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/fde4a00d-016e-4670-a530-f61d1e193a9c/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:384 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/643a86bad1462bd3a707469ddf21c54042779052089d3e793ad1b3b8d40515b7/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/c1c373fbf22263c5b4c058997daa1ca846073c45c132db432e2e279c1fc918b7/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/32ac832785bf5c56348a79a563bbbfca4ef93c86c5a1c2de8e94b930335af7e4/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/b912ee6ef62b7947b45288c6f02b5ccb0f846fdee2bb1c28c4a0738926d5fb8c/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/687de2807d1adde87dfe220631558a3f0738c6809c225f50d6e5085ff273f25b/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/420a4b26e3f25fd140a503c934ed86d7b5fbecd1eb3da2cab8be4b01a5a15773/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/2edbd35e485758ff3c8609653f6b4ce275ec1cd7265474502ddb5e2be838e4c5/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/b9c7db169205b8bd91ac70c89b9305455e351421012985b40383895189f631d2/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/e7e4efecae6c7e225043aa95235f28cca5c4f887a5b459c7b7e3dd7f191d2a09/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/46d444f72321df4268421b0f2535f327ca6bc8c2738cda1da7f2703d1756f38b/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/07d105a3917cd35fbf60d51f390e8c30471baa6d772a8f2ad88c316a1cafe5d5/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-145:{mountpoint:/var/lib/containers/storage/overlay/b9ae548a0d01e78d090bade57d69ba84b7f44b5d9ff961e5bdc4d770d52451a2/merged major:0 minor:145 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/9bbd5921a2de4c8230d69aba698dd9e8277d5e7e58779b353729292760172dff/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/b1d812e4c486d61063a066b69116fc1afac3da1148f85b8b8f2d2e53f9624490/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/51d342424f3a45c4a45e9ebed479da705b68e949267908b551e1d8b10f20f124/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/61686c0d95bc3e84eb5ca4defd6cd781e1cf106d2f6daca263ce7d6c3ce9c2f1/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/f77fe29936d98fa654ce95908717df22d5d8987fc109a5196cdad3967aded4f0/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/5af8435d0ee41a269131ce152a6c47b41b2f8b4ca52706607ba271c7185388a9/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/e063c8f71677a3d5ed8aed11536d6ebe6a32f59b415a070de40ce3254b7670ba/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/c6f973a18c2796175853b26a57e09dad624ed5357a99ffb3bfd0797dd18ed315/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/cac4d1e2b68ffc3bfb0ae383591d309b2aaffbc96bca7c359a9403cb7e2099e7/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/84e5060f9f2e1252343ef299af7911b915c794353721e01496e5a01ed82dd7e1/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/e0feebf60c01e6f473d879d0967e60bc5e60dc524b20c0a1c50a274b875fc62e/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/78b79318403131afbcf9a849155e2defd4e58e93d2cb6ad7927d911db56eed8d/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/a72da4a55f1090c0b58cf2e1c8e3757aaf373f68db5b016187ba8178758b83cf/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/04ee4812bc67d5744fc760c52651225d9cc69fab63791b6910ff8669d394f2eb/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/61a0a994936fe8fc4ef3292eb6f4b1bd5ddf5ddd3e57ee39f63f56b5bbb92e84/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/5034f5332c89b4ce7f83235adbb96bad231b2c47ee623c9814d6397ac02c4866/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/84e3409f9923add36927a808f0883cbf8b183bb72691955f77a98c724efde3cf/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/f3ad62639f30c0477508e0689e440330331f9b0e9d9fb8069fc2249bb738e3f8/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-265:{mountpoint:/var/lib/containers/storage/overlay/3a00008b9882b9b4818dd7d2a3828f220b71b2684db140ee3a29bf1c658fe833/merged major:0 minor:265 fsType:overlay blockSize:0} overlay_0-267:{mountpoint:/var/lib/containers/storage/overlay/3d1b6cb770ec91e5c5c0b394b0191499a991cbd770b2adf8cc51b8d3bd2512d3/merged major:0 minor:267 fsType:overlay blockSize:0} overlay_0-269:{mountpoint:/var/lib/containers/storage/overlay/69122d7eafe131290e92dc9f4d861d8d5c1fa129da44a177304ba513613ad4ce/merged major:0 minor:269 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/7974ae68ed25aafdd60cd432a33d35cfab9a7cf49e2625a880ec62c65ac2edc8/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-277:{mountpoint:/var/lib/containers/storage/overlay/8a66b884b65b8b4470006b7f626f99eeff70acf46f5c41aeaadc22ff40bdaa3e/merged major:0 minor:277 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/f6a7c38784af92b302dfd1f8aded9c244901b1f3ff714b80f6cd16121252d6fb/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/94368620c5f2e4461c2df3952589c538de0af78c94d0c1cdddb5bec817239145/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/81a0579fb0c7b6f010b1e1510a18975101b65651a2b31e7c2ed4c51af105eeed/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/55c0ec0b023a0073b21348f7a4ed1f76b38447d889ddfd01489e980f136a618d/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-299:{mountpoint:/var/lib/containers/storage/overlay/efcd053dd7cc907ddebe66e12f84a954de45715f532fa3d8228e8b14f3202ec1/merged major:0 minor:299 fsType:overlay blockSize:0} overlay_0-305:{mountpoint:/var/lib/containers/storage/overlay/db4ae4722a27be0be6acd60d4253556bf72ea3d9bebe9622e3f30e4ccaaa8459/merged major:0 minor:305 fsType:overlay blockSize:0} overlay_0-322:{mountpoint:/var/lib/containers/storage/overlay/6706dfaa363d299ab3122c884e951ad31e0b52874a5d5cd01e50da8489756bd0/merged major:0 minor:322 fsType:overlay blockSize:0} overlay_0-328:{mountpoint:/var/lib/containers/storage/overlay/84ec51520acce3de151e3825a01f99b2dac37c26671cf6270cbefa944d4467b1/merged major:0 minor:328 fsType:overlay blockSize:0} overlay_0-330:{mountpoint:/var/lib/containers/storage/overlay/8770ab33c71b67f54ff847f9530e5fef86fa65547142daf453fd03e83bf2fffd/merged major:0 minor:330 fsType:overlay blockSize:0} overlay_0-332:{mountpoint:/var/lib/containers/storage/overlay/7c197a4a1026143ee398a733c74b32b1b0984ca76698a07eea8d7224822e31a0/merged major:0 minor:332 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/2b461a4acad78d2d37510b9e30cb85ef04917af6112c58f7cb8d4cba7fc68d71/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-343:{mountpoint:/var/lib/containers/storage/overlay/b55ef693a831bc9d8e8a9d12674b8414a0ad374e45b3d10831726d300bd9f796/merged major:0 minor:343 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/e99bb98e9349c8539b95cff7f7f9af8208f9917bf1f6b4f54cad32d36b728a01/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/cd5d6b0ccd32a05653557881d62465f6eb4ae91d2203fb31fb00da1b58ab8a32/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-359:{mountpoint:/var/lib/containers/storage/overlay/dc31b3f2cbc6dacb20c775406af3c2abe61e47181b745616fedbedce3f5fea48/merged major:0 minor:359 fsType:overlay blockSize:0} overlay_0-381:{mountpoint:/var/lib/containers/storage/overlay/5fed6cd0e8f080b6ec8459fd314944ee13b1967b1821ea94990deed445434234/merged major:0 minor:381 fsType:overlay blockSize:0} overlay_0-392:{mountpoint:/var/lib/containers/storage/overlay/023e2f77ba6fba1cd70ae30d559ef71b1f840906007c04bce3ca16ba543f94ff/merged major:0 minor:392 fsType:overlay blockSize:0} overlay_0-394:{mountpoint:/var/lib/containers/storage/overlay/83b34547fc0ff25ad7bbb621fa000c4f975c3a48f6af455b8194075ddd0995db/merged major:0 minor:394 fsType:overlay blockSize:0} overlay_0-396:{mountpoint:/var/lib/containers/storage/overlay/9da922e264c4c9a651bf1829c44cfb891f36749431212330cd1735a34b50ab34/merged major:0 minor:396 fsType:overlay blockSize:0} overlay_0-398:{mountpoint:/var/lib/containers/storage/overlay/3fc501dc9812813653efce17725855e0ee033b3968fd96267271e74e67120a02/merged major:0 minor:398 fsType:overlay blockSize:0} overlay_0-404:{mountpoint:/var/lib/containers/storage/overlay/c6d47e47b7dc9c5c0dbf29827b2b949b195ddefc6f91056eb31382a7beedb14f/merged major:0 minor:404 fsType:overlay blockSize:0} overlay_0-405:{mountpoint:/var/lib/containers/storage/overlay/4f63de4680bfdda678d6c93185bfb0d25dc20cbc0aa283124cc7dff3ae763f35/merged major:0 minor:405 fsType:overlay blockSize:0} overlay_0-411:{mountpoint:/var/lib/containers/storage/overlay/f7f55976451323b11ed2f00df4abc406849870643286e244a8da1a44ef4a0a34/merged major:0 minor:411 fsType:overlay blockSize:0} overlay_0-416:{mountpoint:/var/lib/containers/storage/overlay/c36e67ac754d89e5c0c2b90079ffacd37d3916a3e33cabb2ffff016a83049629/merged major:0 minor:416 fsType:overlay blockSize:0} overlay_0-421:{mountpoint:/var/lib/containers/storage/overlay/e8f81ec1d286206abc727f8807515f8e0d92209395eee57b0cb2a33d8d1e011d/merged major:0 minor:421 fsType:overlay blockSize:0} overlay_0-426:{mountpoint:/var/lib/containers/storage/overlay/8dd9bee3500cf89cdaad167d1405f85f63eb094e0d7e751ee39542136da5960a/merged major:0 minor:426 fsType:overlay blockSize:0} overlay_0-431:{mountpoint:/var/lib/containers/storage/overlay/f79fd4b79b60599896e6e27bc937a05f86f1657d5551e346025b8a811138abee/merged major:0 minor:431 fsType:overlay blockSize:0} overlay_0-437:{mountpoint:/var/lib/containers/storage/overlay/3e351f70bf61db9e13ac311142f0825d1863f582b7e4fd51d889a9342ed03ae3/merged major:0 minor:437 fsType:overlay blockSize:0} overlay_0-442:{mountpoint:/var/lib/containers/storage/overlay/8cc3056c5152dd1885d2d5937337264ff9f30490e3fe3e05c594b86cbb778586/merged major:0 minor:442 fsType:overlay blockSize:0} overlay_0-64:{mountpoint:/var/lib/containers/storage/overlay/179422058ae0bac5c74f50960fdafedac401f64957ba5ac5f2b5b39c66a1eb95/merged major:0 minor:64 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/ca5c164f49dfb8fe6f5f6fd218ce04dfd302ff1c8ef4092d6fd767485cec3612/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/14c5f58a16e566ea14318f687b779c8149906c40699527939cf384040f0c3416/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/01e2ee313501e56014a347b14d13195348ffab95f7f849fbdc300ca3986d14c7/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/413f8bb25f421bae7549c11c62ea0ed49edea6b744e848572008dd093d1f0a49/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/a3147a858c4c64b53c1cadf8b4a6e496761abce96886a6b1b219371dce88b4a2/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/03fd77506031f3f542e3be06cb2e7b3e0ac87ff4ffed70834d957bc214a8ef02/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/f74b558754b2756b23b889a5976890554995a830bc22e0cf62c724c7beb2cd1d/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/1abd8d8c7721bb34fd0cefc56260bd52c6073a0c1e97afddbbd4e4650f7e75c1/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/32815c9031e628c0be8d634f560ea9e113c15ba4c96de22c62979787556a9f5d/merged major:0 minor:98 fsType:overlay blockSize:0}]
Oct 30 21:34:17.388837 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.388429    5586 manager.go:217] Machine: {Timestamp:2026-10-30 21:34:17.387921795 +0000 UTC m=+0.059825721 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098969 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec298737e412671fd1551da30cfd24af SystemUUID:ec298737-e412-671f-d155-1da30cfd24af BootID:5e6d313a-c2ee-4706-9cbe-228c25cd19f4 Filesystems:[{Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:368 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:376 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-396 DeviceMajor:0 DeviceMinor:396 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:370 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5c24d279-0f41-4c11-b197-c7c9ce7ed028/volumes/kubernetes.io~projected/kube-api-access-fx4xs DeviceMajor:0 DeviceMinor:389 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/76696315-4c5d-4460-a3ab-e027a13211e3/volumes/kubernetes.io~projected/kube-api-access-rwxx8 DeviceMajor:0 DeviceMinor:203 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~projected/kube-api-access-j4jlz DeviceMajor:0 DeviceMinor:315 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:376 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-431 DeviceMajor:0 DeviceMinor:431 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9c2c8b2b422eca5331baa11b3b69649a1681e5ffe0a5df99f9fe2dd8517588df/userdata/shm DeviceMajor:0 DeviceMinor:95 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:364 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:256 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:374 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3142d576-19ca-48d1-b50f-c22945af6ba3/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:48 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e83d89de-6a2a-4950-a82a-d0705739ce81/volumes/kubernetes.io~projected/kube-api-access-9vplh DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/32dd7eaa-b8a6-4de5-a13a-0640b94801e8/volumes/kubernetes.io~projected/kube-api-access-p9mvp DeviceMajor:0 DeviceMinor:52 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bdceec21-dc8d-4813-9637-d9b59174f16f/volumes/kubernetes.io~projected/kube-api-access-4xk77 DeviceMajor:0 DeviceMinor:204 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/917d65d0613fff8e31449f6ec9d568db040849a69afd20001b7f89b436eb3be3/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/783fbfeafd1df21e0299009f71895686501914bd807dc64b8236efefae7bb09b/userdata/shm DeviceMajor:0 DeviceMinor:261 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e603218bcec4ff474ecda24159dea659b950347d7ea0886ebe06d5c16cfa3f2c/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:371 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d74eae5afc9849a57bed32c7be5be8661ec51221d568f49792c77979844689ab/userdata/shm DeviceMajor:0 DeviceMinor:379 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/df91d26b206e05676cd3152cc7a2bedf8c077a01298b4d42b3d273d3834ea190/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:312 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-398 DeviceMajor:0 DeviceMinor:398 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5cbd08bc248d2f9c204a3db6c4a799d7ddeafe5e70f3f4527425fb73b9223c1c/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/33ff5cbba2f296cb9e6a579230ee4feb6697a7e90788051c60d95896bd6e5a6a/userdata/shm DeviceMajor:0 DeviceMinor:83 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b40cbded-e2dc-420d-b454-eec7a903571e/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:243 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6ad46264-9367-4a4e-8eb9-eb0d0470a669/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:383 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-411 DeviceMajor:0 DeviceMinor:411 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/06512914-5083-4ad9-a3da-a6b916f53b6e/volumes/kubernetes.io~projected/kube-api-access-rbc9d DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-64 DeviceMajor:0 DeviceMinor:64 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b40cbded-e2dc-420d-b454-eec7a903571e/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:244 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-267 DeviceMajor:0 DeviceMinor:267 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-305 DeviceMajor:0 DeviceMinor:305 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cd3a25a7a99796cf531c10ebf0bb1c6ed0956feaf3ba31abe2291a30528a721d/userdata/shm DeviceMajor:0 DeviceMinor:326 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-343 DeviceMajor:0 DeviceMinor:343 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8/volumes/kubernetes.io~projected/kube-api-access-s46xr DeviceMajor:0 DeviceMinor:54 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d7cad3ecd7cbdf5061c9f000c7754e145c57d802ba7dac726736f93ac1a2f7dd/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:316 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7f0cf155-c021-465e-a08b-eb2e2aa0935e/volumes/kubernetes.io~projected/kube-api-access-mvjjm DeviceMajor:0 DeviceMinor:325 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:373 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-394 DeviceMajor:0 DeviceMinor:394 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:375 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1414efc0789b873d1e4beb482d7086f82ea661b7b8fe156ee1808d885f1ea5db/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ef43d67f849f8990c0f276df26e38d8460b0390064ed1f7d8aca27a66e157067/userdata/shm DeviceMajor:0 DeviceMinor:253 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:369 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3bfdbb07-013c-4ab3-bf93-05af731a7b13/volumes/kubernetes.io~projected/kube-api-access-4tl6d DeviceMajor:0 DeviceMinor:202 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:372 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-416 DeviceMajor:0 DeviceMinor:416 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c3ed2089-3110-405e-928e-dcdf13e4d4e3/volumes/kubernetes.io~projected/kube-api-access-qjsjw DeviceMajor:0 DeviceMinor:260 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-359 DeviceMajor:0 DeviceMinor:359 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/32dd7eaa-b8a6-4de5-a13a-0640b94801e8/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:47 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/90f49eda-5385-490c-9760-2cf569eabccf/volumes/kubernetes.io~projected/kube-api-access-4h8w4 DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b3ca726df6494fcc0c575a3cd48beb39656c8d74fafcfcabc23968c7fb1941cd/userdata/shm DeviceMajor:0 DeviceMinor:390 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba/volumes/kubernetes.io~projected/kube-api-access-tgfc5 DeviceMajor:0 DeviceMinor:259 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6094eb68105248fad3a349c792a3b857fba7a252a85c8af4d13c346417168a4a/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2ccf87bf-a091-4f22-8811-02dcf621e556/volumes/kubernetes.io~projected/kube-api-access-h5rb7 DeviceMajor:0 DeviceMinor:53 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:258 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:314 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7f0cf155-c021-465e-a08b-eb2e2aa0935e/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:324 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-421 DeviceMajor:0 DeviceMinor:421 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3adefe8b-2ffa-4ee1-aad4-a658970e79aa/volumes/kubernetes.io~projected/kube-api-access-wk97l DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/76696315-4c5d-4460-a3ab-e027a13211e3/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c3ed2089-3110-405e-928e-dcdf13e4d4e3/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:255 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-328 DeviceMajor:0 DeviceMinor:328 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/51baa23e6df870997ec1e1f920801cfa5ab675cc01b830c2dbae35945eca7391/userdata/shm DeviceMajor:0 DeviceMinor:385 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-330 DeviceMajor:0 DeviceMinor:330 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-405 DeviceMajor:0 DeviceMinor:405 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-426 DeviceMajor:0 DeviceMinor:426 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-442 DeviceMajor:0 DeviceMinor:442 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/14cda20d61c1b9e007361243c4d97487f5dd4de58cc27565b4c4d816dbe5ab4c/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:376 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dcbfb3ad0699a53006ea978e770fc8ccbd4d48fe7df3751603bb15d74fa4de82/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:313 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-322 DeviceMajor:0 DeviceMinor:322 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c3ed2089-3110-405e-928e-dcdf13e4d4e3/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:257 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cd58dd4d4967280125b3a50c203b0f98c9312317c5689dab770ad8a25978a729/userdata/shm DeviceMajor:0 DeviceMinor:263 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-381 DeviceMajor:0 DeviceMinor:381 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-437 DeviceMajor:0 DeviceMinor:437 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/32dd7eaa-b8a6-4de5-a13a-0640b94801e8/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:49 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ce77db454de78f3ada0813466ade82455414969ab952277c2fc09bbef09e2a49/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-332 DeviceMajor:0 DeviceMinor:332 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-392 DeviceMajor:0 DeviceMinor:392 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bdceec21-dc8d-4813-9637-d9b59174f16f/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b40cbded-e2dc-420d-b454-eec7a903571e/volumes/kubernetes.io~projected/kube-api-access-kv56g DeviceMajor:0 DeviceMinor:252 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-277 DeviceMajor:0 DeviceMinor:277 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/120cf4e69a67840bf831a546a3fdec252d5e015db1b9d335532e6a01e5c77baa/userdata/shm DeviceMajor:0 DeviceMinor:320 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/6ad46264-9367-4a4e-8eb9-eb0d0470a669/volumes/kubernetes.io~projected/kube-api-access-c246b DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-265 DeviceMajor:0 DeviceMinor:265 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:377 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:376 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/32ac583bb348ae778276066d8e4c1a593d4649ee6535fe01d295c7dd6fbf6d9e/userdata/shm DeviceMajor:0 DeviceMinor:386 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-299 DeviceMajor:0 DeviceMinor:299 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:310 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c5deaa23-062e-431f-bca9-ffb4b60159cd/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:50 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a7fde4cea4342ec8f9c95988990190640238b5389c9ea67da85d41f8364a95c4/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c5deaa23-062e-431f-bca9-ffb4b60159cd/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-269 DeviceMajor:0 DeviceMinor:269 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes/kubernetes.io~projected/kube-api-access-2x4js DeviceMajor:0 DeviceMinor:378 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-145 DeviceMajor:0 DeviceMinor:145 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fde4a00d-016e-4670-a530-f61d1e193a9c/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:384 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-404 DeviceMajor:0 DeviceMinor:404 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c5deaa23-062e-431f-bca9-ffb4b60159cd/volumes/kubernetes.io~projected/kube-api-access-kmwdl DeviceMajor:0 DeviceMinor:51 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3bfdbb07-013c-4ab3-bf93-05af731a7b13/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/17bdd0ae-091c-4442-a4d4-4414e8feb7b2/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:311 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:120cf4e69a67840 MacAddress:d2:0c:68:8c:23:93 Speed:10000 Mtu:8901} {Name:32ac583bb348ae7 MacAddress:86:1e:50:ff:e2:ce Speed:10000 Mtu:8901} {Name:51baa23e6df8709 MacAddress:92:e2:bd:3a:66:08 Speed:10000 Mtu:8901} {Name:6094eb68105248f MacAddress:12:cc:b8:cb:dc:f1 Speed:10000 Mtu:8901} {Name:783fbfeafd1df21 MacAddress:be:49:6a:5e:f7:eb Speed:10000 Mtu:8901} {Name:917d65d0613fff8 MacAddress:6a:75:0d:79:47:dc Speed:10000 Mtu:8901} {Name:b3ca726df6494fc MacAddress:16:50:19:d3:9f:5d Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:44:be:d7:26:6b Speed:0 Mtu:9001} {Name:br-int MacAddress:f2:fb:64:65:5d:59 Speed:0 Mtu:8901} {Name:cd3a25a7a99796c MacAddress:76:c1:76:2d:04:54 Speed:10000 Mtu:8901} {Name:d74eae5afc9849a MacAddress:36:af:5c:ab:5a:e6 Speed:10000 Mtu:8901} {Name:e603218bcec4ff4 MacAddress:c6:7f:f1:27:5e:63 Speed:10000 Mtu:8901} {Name:ef43d67f849f899 MacAddress:2e:2d:00:96:08:93 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:44:be:d7:26:6b Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:96:6f:1f:aa:c9:e0 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:a2:b6:f4:09:c1:93 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:34:17.389055 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.388840    5586 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:34:17.389055 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.388887    5586 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:34:17.389200 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.389175    5586 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:34:17.389351 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.389201    5586 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-104.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:34:17.389396 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.389354    5586 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:34:17.389396 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.389362    5586 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:34:17.389396 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.389372    5586 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:34:17.389396 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.389386    5586 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:34:17.389507 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.389474    5586 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:34:17.389635 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.389626    5586 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:34:17.389743 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.389734    5586 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:34:17.389776 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.389769    5586 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:34:17.389804 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.389781    5586 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:34:17.389804 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.389791    5586 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:34:17.389804 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.389802    5586 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:34:17.389893 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.389809    5586 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:34:17.390459 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.390444    5586 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:34:17.390492 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.390460    5586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:34:17.390956 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.390940    5586 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:34:17.394536 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.394510    5586 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:34:17.394536 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.394536    5586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:34:17.394655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.394644    5586 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 21:34:17.394844 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.394833    5586 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:34:17.394874 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.394863    5586 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:34:17.394999 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.394985    5586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:34:17.395027 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.395012    5586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:34:17.395027 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.395025    5586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:34:17.395084 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.395037    5586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:34:17.395084 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.395049    5586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:34:17.395084 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.395061    5586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:34:17.395084 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.395075    5586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:34:17.395195 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.395086    5586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:34:17.395195 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.395101    5586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:34:17.395195 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.395114    5586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:34:17.395195 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.395142    5586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:34:17.395195 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.395160    5586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:34:17.395345 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.395209    5586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:34:17.395345 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.395221    5586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:34:17.398731 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.398720    5586 server.go:1263] "Started kubelet"
Oct 30 21:34:17.398933 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.398778    5586 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:34:17.398933 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.398847    5586 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:34:17.399151 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.399126    5586 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:34:17.399253 ip-10-0-130-104 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:34:17.400821 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.400732    5586 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:34:17.408697 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.407382    5586 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:34:17.408697 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.407418    5586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:34:17.408697 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.407495    5586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:28:07 +0000 UTC" deadline="2028-05-04 12:02:40.42218639 +0000 UTC"
Oct 30 21:34:17.408697 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.407517    5586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13238h28m23.014672155s"
Oct 30 21:34:17.408697 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.407582    5586 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:34:17.408697 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.407599    5586 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:34:17.408697 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.407669    5586 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:34:17.408697 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:17.407786    5586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-104.us-west-1.compute.internal\" not found"
Oct 30 21:34:17.409710 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.409611    5586 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:34:17.409828 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.409802    5586 factory.go:153] Registering CRI-O factory
Oct 30 21:34:17.409828 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.409820    5586 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.409905    5586 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.409916    5586 factory.go:55] Registering systemd factory
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.409923    5586 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.409941    5586 factory.go:103] Registering Raw factory
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.409957    5586 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410050    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17bdd0ae-091c-4442-a4d4-4414e8feb7b2" volumeName="kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410104    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b40cbded-e2dc-420d-b454-eec7a903571e" volumeName="kubernetes.io/secret/b40cbded-e2dc-420d-b454-eec7a903571e-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410118    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdceec21-dc8d-4813-9637-d9b59174f16f" volumeName="kubernetes.io/projected/bdceec21-dc8d-4813-9637-d9b59174f16f-kube-api-access-4xk77" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410130    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410141    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" volumeName="kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410153    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c3ed2089-3110-405e-928e-dcdf13e4d4e3" volumeName="kubernetes.io/empty-dir/c3ed2089-3110-405e-928e-dcdf13e4d4e3-volume-directive-shadow" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410164    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c5deaa23-062e-431f-bca9-ffb4b60159cd" volumeName="kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410176    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410189    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-tls-assets" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410201    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-metrics-client-certs" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410213    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90f49eda-5385-490c-9760-2cf569eabccf" volumeName="kubernetes.io/configmap/90f49eda-5385-490c-9760-2cf569eabccf-multus-daemon-config" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410224    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410251    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-config" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410268    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17bdd0ae-091c-4442-a4d4-4414e8feb7b2" volumeName="kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410283    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32dd7eaa-b8a6-4de5-a13a-0640b94801e8" volumeName="kubernetes.io/empty-dir/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-tuned" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410295    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdceec21-dc8d-4813-9637-d9b59174f16f" volumeName="kubernetes.io/configmap/bdceec21-dc8d-4813-9637-d9b59174f16f-config-volume" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410301    5586 manager.go:319] Starting recovery of all containers
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410306    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-tls" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410332    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" volumeName="kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410344    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bfdbb07-013c-4ab3-bf93-05af731a7b13" volumeName="kubernetes.io/projected/3bfdbb07-013c-4ab3-bf93-05af731a7b13-kube-api-access-4tl6d" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410357    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410369    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410380    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e83d89de-6a2a-4950-a82a-d0705739ce81" volumeName="kubernetes.io/projected/e83d89de-6a2a-4950-a82a-d0705739ce81-kube-api-access-9vplh" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410392    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90f49eda-5385-490c-9760-2cf569eabccf" volumeName="kubernetes.io/configmap/90f49eda-5385-490c-9760-2cf569eabccf-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410404    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90f49eda-5385-490c-9760-2cf569eabccf" volumeName="kubernetes.io/projected/90f49eda-5385-490c-9760-2cf569eabccf-kube-api-access-4h8w4" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410415    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c3ed2089-3110-405e-928e-dcdf13e4d4e3" volumeName="kubernetes.io/configmap/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410428    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410455    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410467    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3142d576-19ca-48d1-b50f-c22945af6ba3" volumeName="kubernetes.io/secret/3142d576-19ca-48d1-b50f-c22945af6ba3-agent-certs" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410498    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c3ed2089-3110-405e-928e-dcdf13e4d4e3" volumeName="kubernetes.io/projected/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-api-access-qjsjw" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410513    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-grpc-tls" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410523    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-web-config" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410537    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06512914-5083-4ad9-a3da-a6b916f53b6e" volumeName="kubernetes.io/configmap/06512914-5083-4ad9-a3da-a6b916f53b6e-serviceca" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410548    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06512914-5083-4ad9-a3da-a6b916f53b6e" volumeName="kubernetes.io/projected/06512914-5083-4ad9-a3da-a6b916f53b6e-kube-api-access-rbc9d" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410560    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bfdbb07-013c-4ab3-bf93-05af731a7b13" volumeName="kubernetes.io/secret/3bfdbb07-013c-4ab3-bf93-05af731a7b13-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410571    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76696315-4c5d-4460-a3ab-e027a13211e3" volumeName="kubernetes.io/projected/76696315-4c5d-4460-a3ab-e027a13211e3-kube-api-access-rwxx8" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410583    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7f0cf155-c021-465e-a08b-eb2e2aa0935e" volumeName="kubernetes.io/projected/7f0cf155-c021-465e-a08b-eb2e2aa0935e-kube-api-access-mvjjm" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410595    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c5deaa23-062e-431f-bca9-ffb4b60159cd" volumeName="kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-env-overrides" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410608    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c5deaa23-062e-431f-bca9-ffb4b60159cd" volumeName="kubernetes.io/projected/c5deaa23-062e-431f-bca9-ffb4b60159cd-kube-api-access-kmwdl" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410625    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17bdd0ae-091c-4442-a4d4-4414e8feb7b2" volumeName="kubernetes.io/projected/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-kube-api-access-j4jlz" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410646    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42caab53-09ce-4c8f-b7ab-5d86ae4b64ba" volumeName="kubernetes.io/secret/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-tls" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410669    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ad46264-9367-4a4e-8eb9-eb0d0470a669" volumeName="kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410680    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c5deaa23-062e-431f-bca9-ffb4b60159cd" volumeName="kubernetes.io/secret/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410692    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410704    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410716    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17bdd0ae-091c-4442-a4d4-4414e8feb7b2" volumeName="kubernetes.io/configmap/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410729    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17bdd0ae-091c-4442-a4d4-4414e8feb7b2" volumeName="kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410741    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17bdd0ae-091c-4442-a4d4-4414e8feb7b2" volumeName="kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410751    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bfdbb07-013c-4ab3-bf93-05af731a7b13" volumeName="kubernetes.io/empty-dir/3bfdbb07-013c-4ab3-bf93-05af731a7b13-data-volume" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410763    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42caab53-09ce-4c8f-b7ab-5d86ae4b64ba" volumeName="kubernetes.io/configmap/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410775    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ad46264-9367-4a4e-8eb9-eb0d0470a669" volumeName="kubernetes.io/projected/6ad46264-9367-4a4e-8eb9-eb0d0470a669-kube-api-access-c246b" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410788    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b40cbded-e2dc-420d-b454-eec7a903571e" volumeName="kubernetes.io/configmap/b40cbded-e2dc-420d-b454-eec7a903571e-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410801    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b40cbded-e2dc-420d-b454-eec7a903571e" volumeName="kubernetes.io/projected/b40cbded-e2dc-420d-b454-eec7a903571e-kube-api-access-kv56g" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410813    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42caab53-09ce-4c8f-b7ab-5d86ae4b64ba" volumeName="kubernetes.io/configmap/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410825    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42caab53-09ce-4c8f-b7ab-5d86ae4b64ba" volumeName="kubernetes.io/secret/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410837    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76696315-4c5d-4460-a3ab-e027a13211e3" volumeName="kubernetes.io/secret/76696315-4c5d-4460-a3ab-e027a13211e3-cert" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410850    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-kube-api-access-2x4js" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410872    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17bdd0ae-091c-4442-a4d4-4414e8feb7b2" volumeName="kubernetes.io/empty-dir/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-tmp" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410886    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17bdd0ae-091c-4442-a4d4-4414e8feb7b2" volumeName="kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410902    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" volumeName="kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410914    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42caab53-09ce-4c8f-b7ab-5d86ae4b64ba" volumeName="kubernetes.io/projected/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-kube-api-access-tgfc5" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410925    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b40cbded-e2dc-420d-b454-eec7a903571e" volumeName="kubernetes.io/secret/b40cbded-e2dc-420d-b454-eec7a903571e-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410939    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c3ed2089-3110-405e-928e-dcdf13e4d4e3" volumeName="kubernetes.io/secret/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410952    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c5deaa23-062e-431f-bca9-ffb4b60159cd" volumeName="kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovnkube-config" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410966    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410979    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3142d576-19ca-48d1-b50f-c22945af6ba3" volumeName="kubernetes.io/configmap/3142d576-19ca-48d1-b50f-c22945af6ba3-konnectivity-ca" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.410994    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32dd7eaa-b8a6-4de5-a13a-0640b94801e8" volumeName="kubernetes.io/empty-dir/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-tmp" seLinuxMountContext=""
Oct 30 21:34:17.410954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411006    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdceec21-dc8d-4813-9637-d9b59174f16f" volumeName="kubernetes.io/secret/bdceec21-dc8d-4813-9637-d9b59174f16f-metrics-tls" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411022    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-config-out" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411035    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-thanos-prometheus-http-client-file" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411047    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8" volumeName="kubernetes.io/empty-dir/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-tmp-dir" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411058    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8" volumeName="kubernetes.io/projected/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-kube-api-access-s46xr" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411070    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fde4a00d-016e-4670-a530-f61d1e193a9c" volumeName="kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411086    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ccf87bf-a091-4f22-8811-02dcf621e556" volumeName="kubernetes.io/configmap/2ccf87bf-a091-4f22-8811-02dcf621e556-iptables-alerter-script" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411101    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ccf87bf-a091-4f22-8811-02dcf621e556" volumeName="kubernetes.io/projected/2ccf87bf-a091-4f22-8811-02dcf621e556-kube-api-access-h5rb7" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411113    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32dd7eaa-b8a6-4de5-a13a-0640b94801e8" volumeName="kubernetes.io/projected/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-kube-api-access-p9mvp" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411125    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" volumeName="kubernetes.io/projected/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-kube-api-access-wk97l" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411137    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bfdbb07-013c-4ab3-bf93-05af731a7b13" volumeName="kubernetes.io/configmap/3bfdbb07-013c-4ab3-bf93-05af731a7b13-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411150    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5c24d279-0f41-4c11-b197-c7c9ce7ed028" volumeName="kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411163    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdceec21-dc8d-4813-9637-d9b59174f16f" volumeName="kubernetes.io/empty-dir/bdceec21-dc8d-4813-9637-d9b59174f16f-tmp-dir" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411177    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c3ed2089-3110-405e-928e-dcdf13e4d4e3" volumeName="kubernetes.io/configmap/c3ed2089-3110-405e-928e-dcdf13e4d4e3-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411192    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17bdd0ae-091c-4442-a4d4-4414e8feb7b2" volumeName="kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-grpc-tls" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411206    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7f0cf155-c021-465e-a08b-eb2e2aa0935e" volumeName="kubernetes.io/secret/7f0cf155-c021-465e-a08b-eb2e2aa0935e-monitoring-plugin-cert" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411221    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0534933-54d7-47c4-9302-523383fa1fc1" volumeName="kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-db" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411248    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42caab53-09ce-4c8f-b7ab-5d86ae4b64ba" volumeName="kubernetes.io/empty-dir/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-textfile" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411260    5586 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c3ed2089-3110-405e-928e-dcdf13e4d4e3" volumeName="kubernetes.io/secret/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-tls" seLinuxMountContext=""
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411273    5586 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:34:17.415461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.411282    5586 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:34:17.435431 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.435376    5586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:34:17.439350 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.439323    5586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:34:17.439350 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.439348    5586 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:34:17.439502 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.439371    5586 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:34:17.439502 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:17.439430    5586 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 21:34:17.452723 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.452675    5586 generic.go:356] "Generic (PLEG): container finished" podID="42caab53-09ce-4c8f-b7ab-5d86ae4b64ba" containerID="289f1b7dcc64f924612f95ee3cdb443eb7258d0902b4480a51dd0532db2792c7" exitCode=0
Oct 30 21:34:17.460063 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.460031    5586 generic.go:356] "Generic (PLEG): container finished" podID="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" containerID="e963196f920a806acf814a2ad782f745eaec5a80f3fc2a75aa3ca92e733a110c" exitCode=0
Oct 30 21:34:17.460063 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.460055    5586 generic.go:356] "Generic (PLEG): container finished" podID="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" containerID="92d4c910b1e531353a66006ea8e6aa8903a713206a4202289ea61282f392d3cc" exitCode=0
Oct 30 21:34:17.460063 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.460063    5586 generic.go:356] "Generic (PLEG): container finished" podID="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" containerID="2991e61a11e7fb3fa3ac422f788e510f32ba7dca81f64e65ee479126df5672ec" exitCode=0
Oct 30 21:34:17.460063 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.460071    5586 generic.go:356] "Generic (PLEG): container finished" podID="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" containerID="cf6df429750269077117dc72dfd54070b78d4ffc9ab2f4c1b7d3304938f5e763" exitCode=0
Oct 30 21:34:17.460420 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.460077    5586 generic.go:356] "Generic (PLEG): container finished" podID="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" containerID="9942b46ba4e9c7adf714f41361b19b26a58462cadd887ed40fc050c1e821bcb2" exitCode=0
Oct 30 21:34:17.460420 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.460083    5586 generic.go:356] "Generic (PLEG): container finished" podID="3adefe8b-2ffa-4ee1-aad4-a658970e79aa" containerID="06305bf6ac59de3ec94a0092661ba39f391f56f0f31892935b3df5cb3cd7ceef" exitCode=0
Oct 30 21:34:17.491004 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.490964    5586 generic.go:356] "Generic (PLEG): container finished" podID="8eb924ba35b0f8556b9fd18e228cc9da" containerID="3b266933a07282503efa61a74ae22378f84cf2681e028d7b8711e13686a0c950" exitCode=0
Oct 30 21:34:17.498038 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.498010    5586 generic.go:356] "Generic (PLEG): container finished" podID="d0534933-54d7-47c4-9302-523383fa1fc1" containerID="76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49" exitCode=0
Oct 30 21:34:17.507910 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:17.507830    5586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-104.us-west-1.compute.internal\" not found"
Oct 30 21:34:17.522160 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.522137    5586 manager.go:324] Recovery completed
Oct 30 21:34:17.539653 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:17.539607    5586 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 21:34:17.544612 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.544596    5586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:34:17.546320 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.546295    5586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:34:17.546402 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.546326    5586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:34:17.546402 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.546337    5586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:34:17.549325 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.549304    5586 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:34:17.549421 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.549339    5586 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:34:17.549804 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.549790    5586 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:34:17.549834 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.549803    5586 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:34:17.549834 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.549823    5586 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:34:17.549929 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.549855    5586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:34:17.549981 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.549948    5586 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 21:34:17.549981 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.549959    5586 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 21:34:17.549981 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.549976    5586 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 21:34:17.549981 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.549982    5586 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 21:34:17.550107 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.549988    5586 policy_none.go:50] "Start"
Oct 30 21:34:17.550107 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.549996    5586 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:34:17.550107 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.550006    5586 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:34:17.550203 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.550191    5586 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 21:34:17.550257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.550202    5586 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 21:34:17.550257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.550208    5586 policy_none.go:44] "Start"
Oct 30 21:34:17.550791 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.550774    5586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:34:17.550873 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.550806    5586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:34:17.550873 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.550822    5586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:34:17.554874 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.554847    5586 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:34:17.557485 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.557463    5586 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:34:17.557571 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:17.557487    5586 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:34:17.557571 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.557496    5586 server.go:87] "Starting device plugin registration server"
Oct 30 21:34:17.557753 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.557740    5586 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:34:17.558371 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.557753    5586 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:34:17.558469 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.558392    5586 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:34:17.558527 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.558486    5586 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:34:17.558527 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.558517    5586 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:34:17.558527 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.558523    5586 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:34:17.558683 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.558572    5586 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:34:17.558506874Z","UUID":"9fefe2d0-3f2e-4d94-a085-2249410f9e87","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:34:17.559663 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:17.559641    5586 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:34:17.559744 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:17.559679    5586 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-104.us-west-1.compute.internal\" not found"
Oct 30 21:34:17.560028 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.560008    5586 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:34:17.560121 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.560038    5586 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:34:17.645993 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.645955    5586 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:34:17.646353 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.646335    5586 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 21:34:17.659307 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.659289    5586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:34:17.660563 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.660545    5586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:34:17.660621 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.660576    5586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:34:17.660621 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.660587    5586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-104.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:34:17.660706 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.660629    5586 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.696105 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.696080    5586 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:34:17.740695 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.740620    5586 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"]
Oct 30 21:34:17.741382 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.741344    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal" event={"ID":"8eb924ba35b0f8556b9fd18e228cc9da","Type":"ContainerStarted","Data":"b2767587452130ada8015d789f303a81bcbc55b5aa1d94e2f78636fe5809e927"}
Oct 30 21:34:17.741486 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.741389    5586 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.741569 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.741392    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal" event={"ID":"8eb924ba35b0f8556b9fd18e228cc9da","Type":"ContainerDied","Data":"3b266933a07282503efa61a74ae22378f84cf2681e028d7b8711e13686a0c950"}
Oct 30 21:34:17.741569 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.741512    5586 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.741569 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.741542    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal" event={"ID":"8eb924ba35b0f8556b9fd18e228cc9da","Type":"ContainerStarted","Data":"1414efc0789b873d1e4beb482d7086f82ea661b7b8fe156ee1808d885f1ea5db"}
Oct 30 21:34:17.741569 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.741565    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal" event={"ID":"6ea3e08f52bca920602c76ad35bcaf69","Type":"ContainerStarted","Data":"5793af378c29fe1e044424dd6e8703df12d4e595fb89c7fe490c34be53601f13"}
Oct 30 21:34:17.741780 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.741579    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal" event={"ID":"6ea3e08f52bca920602c76ad35bcaf69","Type":"ContainerStarted","Data":"dcbfb3ad0699a53006ea978e770fc8ccbd4d48fe7df3751603bb15d74fa4de82"}
Oct 30 21:34:17.801158 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.801073    5586 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:34:17.801158 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:17.801135    5586 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.803092 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.803071    5586 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:34:17.803290 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:17.803116    5586 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.803452 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.803435    5586 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.803506 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.803497    5586 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.847007 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.846965    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8eb924ba35b0f8556b9fd18e228cc9da-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"8eb924ba35b0f8556b9fd18e228cc9da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.847007 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.847010    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8eb924ba35b0f8556b9fd18e228cc9da-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"8eb924ba35b0f8556b9fd18e228cc9da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.847205 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.847030    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ea3e08f52bca920602c76ad35bcaf69-config\") pod \"kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"6ea3e08f52bca920602c76ad35bcaf69\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.948034 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.947996    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8eb924ba35b0f8556b9fd18e228cc9da-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"8eb924ba35b0f8556b9fd18e228cc9da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.948034 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.948030    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8eb924ba35b0f8556b9fd18e228cc9da-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"8eb924ba35b0f8556b9fd18e228cc9da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.948317 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.948052    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ea3e08f52bca920602c76ad35bcaf69-config\") pod \"kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"6ea3e08f52bca920602c76ad35bcaf69\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.948317 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.948113    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8eb924ba35b0f8556b9fd18e228cc9da-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"8eb924ba35b0f8556b9fd18e228cc9da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.948317 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.948151    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ea3e08f52bca920602c76ad35bcaf69-config\") pod \"kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"6ea3e08f52bca920602c76ad35bcaf69\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:17.948317 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:17.948116    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8eb924ba35b0f8556b9fd18e228cc9da-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal\" (UID: \"8eb924ba35b0f8556b9fd18e228cc9da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:18.391016 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.390964    5586 apiserver.go:52] "Watching apiserver"
Oct 30 21:34:18.493845 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.493807    5586 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:34:18.494514 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.494486    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-jgphd","openshift-network-operator/iptables-alerter-wdcq5","kube-system/global-pull-secret-syncer-ztncc","openshift-dns/dns-default-5svzl","openshift-monitoring/node-exporter-zts42","openshift-multus/multus-2m72k","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal","openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck","openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s","openshift-dns/node-resolver-z8g8q","openshift-monitoring/monitoring-plugin-77f4f76594-btfmp","openshift-network-diagnostics/network-check-target-2r74t","openshift-ovn-kubernetes/ovnkube-node-jfbvs","kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal","openshift-cluster-node-tuning-operator/tuned-xrkmw","openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc","kube-system/konnectivity-agent-9zq7q","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px","openshift-monitoring/prometheus-k8s-0","openshift-multus/network-metrics-daemon-w9rmz","openshift-insights/insights-runtime-extractor-csjtg","openshift-ingress-canary/ingress-canary-dskw4","openshift-multus/multus-additional-cni-plugins-kx4s8"]
Oct 30 21:34:18.506455 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.506421    5586 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:18.506650 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.506539    5586 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:18.549673 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.549636    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:34:18.549673 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.549673    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:34:18.549673 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.549682    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:34:18.549977 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.549689    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:34:18.549977 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.549701    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:34:18.549977 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.549736    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:34:18.549977 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.549858    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:34:18.549977 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.549861    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:34:18.549977 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.549978    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:34:18.550337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.549988    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:34:18.550337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.549995    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:34:18.550337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550043    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:34:18.550337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550133    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:34:18.550337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550196    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mzgn4\""
Oct 30 21:34:18.550337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550198    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:34:18.550337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550252    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:34:18.550337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550250    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-zs2mb\""
Oct 30 21:34:18.550337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550299    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rjmnn\""
Oct 30 21:34:18.550337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550283    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:34:18.550337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550325    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:34:18.550811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550257    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:34:18.550811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550366    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.550811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550304    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:34:18.550811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550335    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-qlt7z\""
Oct 30 21:34:18.550811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550375    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jnpcj\""
Oct 30 21:34:18.550811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550371    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:34:18.550811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550491    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3bfdbb07-013c-4ab3-bf93-05af731a7b13-crio-socket\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:34:18.550811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550540    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b40cbded-e2dc-420d-b454-eec7a903571e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:34:18.550811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550569    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tvlkp\""
Oct 30 21:34:18.550811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550613    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-76thg\""
Oct 30 21:34:18.550811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550652    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.550811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550805    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-env-overrides\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550653    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-wn5fv\""
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550838    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3bfdbb07-013c-4ab3-bf93-05af731a7b13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550682    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2v7jg\""
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550681    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550724    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550746    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550694    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550816    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550726    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6t698\""
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550887    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.550874    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-multus-certs\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551034    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3142d576-19ca-48d1-b50f-c22945af6ba3-agent-certs\") pod \"konnectivity-agent-9zq7q\" (UID: \"3142d576-19ca-48d1-b50f-c22945af6ba3\") " pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551064    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551084    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-env-overrides\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551094    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.551190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551188    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdceec21-dc8d-4813-9637-d9b59174f16f-metrics-tls\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551212    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551326    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551334    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cni-binary-copy\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551343    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3142d576-19ca-48d1-b50f-c22945af6ba3-agent-certs\") pod \"konnectivity-agent-9zq7q\" (UID: \"3142d576-19ca-48d1-b50f-c22945af6ba3\") " pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551362    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-ovn\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551390    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551415    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-cni-bin\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551438    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovnkube-config\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551498    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovnkube-script-lib\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551589    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmwdl\" (UniqueName: \"kubernetes.io/projected/c5deaa23-062e-431f-bca9-ffb4b60159cd-kube-api-access-kmwdl\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551592    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovnkube-config\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551606    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cnibin\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551623    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551640    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-registration-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.551664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551655    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-run\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551672    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-metrics-client-ca\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551695    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovnkube-script-lib\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551746    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551768    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551791    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4jlz\" (UniqueName: \"kubernetes.io/projected/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-kube-api-access-j4jlz\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551819    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-var-lib-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551846    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3bfdbb07-013c-4ab3-bf93-05af731a7b13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551849    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551893    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-cni-bin\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551918    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysctl-d\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551924    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.551982    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv56g\" (UniqueName: \"kubernetes.io/projected/b40cbded-e2dc-420d-b454-eec7a903571e-kube-api-access-kv56g\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552017    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-accelerators-collector-config\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552045    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xk77\" (UniqueName: \"kubernetes.io/projected/bdceec21-dc8d-4813-9637-d9b59174f16f-kube-api-access-4xk77\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552073    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3ed2089-3110-405e-928e-dcdf13e4d4e3-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552098    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-config-out\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552122    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90f49eda-5385-490c-9760-2cf569eabccf-cni-binary-copy\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552151    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-cni-multus\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552178    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-var-lib-kubelet\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552213    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-metrics-client-ca\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.552283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552278    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-node-log\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552291    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-config-out\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552316    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2ccf87bf-a091-4f22-8811-02dcf621e556-iptables-alerter-script\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552333    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90f49eda-5385-490c-9760-2cf569eabccf-cni-binary-copy\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552352    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-kubernetes\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552373    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552410    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5rb7\" (UniqueName: \"kubernetes.io/projected/2ccf87bf-a091-4f22-8811-02dcf621e556-kube-api-access-h5rb7\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552472    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk97l\" (UniqueName: \"kubernetes.io/projected/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-kube-api-access-wk97l\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552494    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552504    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2ccf87bf-a091-4f22-8811-02dcf621e556-iptables-alerter-script\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552559    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552588    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-tmp\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552612    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552637    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c3ed2089-3110-405e-928e-dcdf13e4d4e3-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552670    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjsjw\" (UniqueName: \"kubernetes.io/projected/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-api-access-qjsjw\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552693    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-tmp\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552709    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-tmp\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552748    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552769    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-run-netns\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552767    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-tmp\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552760    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c3ed2089-3110-405e-928e-dcdf13e4d4e3-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552822    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-conf-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552840    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-lib-modules\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552877    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-etc-selinux\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552906    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7f0cf155-c021-465e-a08b-eb2e2aa0935e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-btfmp\" (UID: \"7f0cf155-c021-465e-a08b-eb2e2aa0935e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552928    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-config\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552955    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552980    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06512914-5083-4ad9-a3da-a6b916f53b6e-host\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.552997    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.553094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553093    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-kubelet\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553182    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbc9d\" (UniqueName: \"kubernetes.io/projected/06512914-5083-4ad9-a3da-a6b916f53b6e-kube-api-access-rbc9d\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553211    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553252    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-web-config\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553277    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553320    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553348    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553375    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-systemd-units\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553395    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-k8s-cni-cncf-io\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553414    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553432    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-socket-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553483    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553616    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553655    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553732    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fde4a00d-016e-4670-a530-f61d1e193a9c-original-pull-secret\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553750    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-socket-dir-parent\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553781    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-kubelet\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553800    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/06512914-5083-4ad9-a3da-a6b916f53b6e-serviceca\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553817    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-sys\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553833    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3142d576-19ca-48d1-b50f-c22945af6ba3-konnectivity-ca\") pod \"konnectivity-agent-9zq7q\" (UID: \"3142d576-19ca-48d1-b50f-c22945af6ba3\") " pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553849    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-wtmp\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553878    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvjjm\" (UniqueName: \"kubernetes.io/projected/7f0cf155-c021-465e-a08b-eb2e2aa0935e-kube-api-access-mvjjm\") pod \"monitoring-plugin-77f4f76594-btfmp\" (UID: \"7f0cf155-c021-465e-a08b-eb2e2aa0935e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553907    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-netns\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553956    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c246b\" (UniqueName: \"kubernetes.io/projected/6ad46264-9367-4a4e-8eb9-eb0d0470a669-kube-api-access-c246b\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.553998    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysconfig\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554025    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3142d576-19ca-48d1-b50f-c22945af6ba3-konnectivity-ca\") pod \"konnectivity-agent-9zq7q\" (UID: \"3142d576-19ca-48d1-b50f-c22945af6ba3\") " pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554026    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgfc5\" (UniqueName: \"kubernetes.io/projected/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-kube-api-access-tgfc5\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554057    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554100    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90f49eda-5385-490c-9760-2cf569eabccf-multus-daemon-config\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554118    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-etc-kubernetes\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554145    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h8w4\" (UniqueName: \"kubernetes.io/projected/90f49eda-5385-490c-9760-2cf569eabccf-kube-api-access-4h8w4\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554174    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx4xs\" (UniqueName: \"kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs\") pod \"network-check-target-2r74t\" (UID: \"5c24d279-0f41-4c11-b197-c7c9ce7ed028\") " pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554207    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-tls\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554248    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-cnibin\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554275    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-systemd\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554300    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90f49eda-5385-490c-9760-2cf569eabccf-multus-daemon-config\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554314    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76696315-4c5d-4460-a3ab-e027a13211e3-cert\") pod \"ingress-canary-dskw4\" (UID: \"76696315-4c5d-4460-a3ab-e027a13211e3\") " pod="openshift-ingress-canary/ingress-canary-dskw4"
Oct 30 21:34:18.554354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554332    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-grpc-tls\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554411    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-slash\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554429    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-hosts-file\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554468    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-log-socket\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554497    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovn-node-metrics-cert\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554524    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b40cbded-e2dc-420d-b454-eec7a903571e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554523    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/06512914-5083-4ad9-a3da-a6b916f53b6e-serviceca\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554667    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-textfile\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554689    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-sys-fs\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554718    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x4js\" (UniqueName: \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-kube-api-access-2x4js\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554715    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c5deaa23-062e-431f-bca9-ffb4b60159cd-ovn-node-metrics-cert\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554763    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-textfile\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554771    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554807    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-cni-netd\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554844    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554879    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-modprobe-d\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554910    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fde4a00d-016e-4670-a530-f61d1e193a9c-kubelet-config\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554956    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.554989    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bdceec21-dc8d-4813-9637-d9b59174f16f-config-volume\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555017    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-tmp-dir\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555051    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-cni-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555064    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-tmp-dir\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555081    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9mvp\" (UniqueName: \"kubernetes.io/projected/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-kube-api-access-p9mvp\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555094    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ad46264-9367-4a4e-8eb9-eb0d0470a669-metrics-certs\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555109    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b40cbded-e2dc-420d-b454-eec7a903571e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555141    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-device-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555158    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-etc-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555175    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s46xr\" (UniqueName: \"kubernetes.io/projected/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-kube-api-access-s46xr\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555194    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-hostroot\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555228    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ccf87bf-a091-4f22-8811-02dcf621e556-host-slash\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555281    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tl6d\" (UniqueName: \"kubernetes.io/projected/3bfdbb07-013c-4ab3-bf93-05af731a7b13-kube-api-access-4tl6d\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555334    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-tuned\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555401    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-tuned\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555428    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-system-cni-dir\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555462    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555478    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555501    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysctl-conf\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555525    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-root\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555595    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555631    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bfdbb07-013c-4ab3-bf93-05af731a7b13-data-volume\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555666    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-host\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555683    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-sys\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555682    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555734    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fde4a00d-016e-4670-a530-f61d1e193a9c-dbus\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555767    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bfdbb07-013c-4ab3-bf93-05af731a7b13-data-volume\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555807    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-tls\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555861    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vplh\" (UniqueName: \"kubernetes.io/projected/e83d89de-6a2a-4950-a82a-d0705739ce81-kube-api-access-9vplh\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555882    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwxx8\" (UniqueName: \"kubernetes.io/projected/76696315-4c5d-4460-a3ab-e027a13211e3-kube-api-access-rwxx8\") pod \"ingress-canary-dskw4\" (UID: \"76696315-4c5d-4460-a3ab-e027a13211e3\") " pod="openshift-ingress-canary/ingress-canary-dskw4"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555909    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-system-cni-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.555907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555938    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdceec21-dc8d-4813-9637-d9b59174f16f-tmp-dir\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:34:18.557280 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.555966    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.557280 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.556016    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-systemd\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.557280 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.556059    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdceec21-dc8d-4813-9637-d9b59174f16f-tmp-dir\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:34:18.596208 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.596175    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:34:18.596208 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.596177    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pplxq\""
Oct 30 21:34:18.596416 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.596326    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pczpt\""
Oct 30 21:34:18.596416 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.596350    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:34:18.596508 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.596422    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-cprz9\""
Oct 30 21:34:18.596508 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.596430    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-26jx5\""
Oct 30 21:34:18.601554 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.601535    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3bfdbb07-013c-4ab3-bf93-05af731a7b13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:34:18.604016 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.603996    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 21:34:18.604111 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.604094    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:34:18.604305 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.604283    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 21:34:18.604367 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.604348    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:34:18.604553 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.604542    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:34:18.604689 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.604674    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:34:18.604736 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.604712    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:34:18.605017 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605001    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:34:18.605092 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605035    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:34:18.605092 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605050    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:34:18.605092 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605070    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:34:18.605257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605118    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:34:18.605257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605145    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:34:18.605257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605186    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 21:34:18.605257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605184    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:34:18.605257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605203    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:34:18.605437 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605270    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bdceec21-dc8d-4813-9637-d9b59174f16f-config-volume\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:34:18.605437 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605273    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:34:18.605437 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605417    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:34:18.605520 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605450    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:34:18.605636 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605623    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:34:18.605670 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605644    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 21:34:18.605779 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.605760    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 21:34:18.606214 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.606187    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-flltm\""
Oct 30 21:34:18.606325 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.606250    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 21:34:18.606325 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.606268    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 21:34:18.606325 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.606273    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:34:18.606450 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.606408    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 21:34:18.606507 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.606490    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:34:18.606892 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.606870    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:34:18.606992 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.606973    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 21:34:18.607194 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.607179    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 21:34:18.607374 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.607359    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2kj04h3u19ra3\""
Oct 30 21:34:18.607668 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.607653    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 21:34:18.610409 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.607894    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1gtgf37ikplq8\""
Oct 30 21:34:18.610409 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.607979    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 21:34:18.610409 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.607997    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 21:34:18.610409 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.608086    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-bxch5\""
Oct 30 21:34:18.610409 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.608108    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 21:34:18.610409 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.608140    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:34:18.610409 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.608410    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l2n6f\""
Oct 30 21:34:18.610409 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.608593    5586 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:34:18.610750 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.610725    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.611314 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.611297    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b40cbded-e2dc-420d-b454-eec7a903571e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:34:18.611431 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.611415    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.611554 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.611491    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.611671 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.611657    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.611941 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.611925    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-metrics-client-ca\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.611993 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.611975    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:34:18.612090 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.612073    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3bfdbb07-013c-4ab3-bf93-05af731a7b13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:34:18.612228 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.612213    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3ed2089-3110-405e-928e-dcdf13e4d4e3-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:34:18.612299 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.612224    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-accelerators-collector-config\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.612418 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.612403    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-metrics-client-ca\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.612835 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.612818    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.612886 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.612822    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.614090 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.614005    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.614090 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.614021    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.614090 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.614020    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.614090 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.614073    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-web-config\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.614281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.614071    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.614281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.614103    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.614281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.614125    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.614281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.614125    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.614528 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.614512    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-tls\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.614585 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.614515    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76696315-4c5d-4460-a3ab-e027a13211e3-cert\") pod \"ingress-canary-dskw4\" (UID: \"76696315-4c5d-4460-a3ab-e027a13211e3\") " pod="openshift-ingress-canary/ingress-canary-dskw4"
Oct 30 21:34:18.614618 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.614604    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-grpc-tls\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.614753 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.614740    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b40cbded-e2dc-420d-b454-eec7a903571e-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:34:18.616436 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.616418    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.616477 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.616450    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b40cbded-e2dc-420d-b454-eec7a903571e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:34:18.645571 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.645502    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 21:34:18.645571 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.645548    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 21:34:18.645815 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.645802    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 21:34:18.645915 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.645814    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 21:34:18.645915 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.645821    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.646023 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.646010    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-njnn9\""
Oct 30 21:34:18.646068 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.646021    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-th4t8\""
Oct 30 21:34:18.646949 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.646932    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 21:34:18.646997 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.646950    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 21:34:18.647180 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.647167    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tvbqn\""
Oct 30 21:34:18.652295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.652277    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.653592 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.653578    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.653661 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.653577    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7f0cf155-c021-465e-a08b-eb2e2aa0935e-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-btfmp\" (UID: \"7f0cf155-c021-465e-a08b-eb2e2aa0935e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:18.653719 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.653701    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-config\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.653947 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.653934    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-xxswp\""
Oct 30 21:34:18.654007 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.653984    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:34:18.654336 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.654321    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.654603 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.654564    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 21:34:18.654603 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.654579    5586 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:34:18.654717 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:18.654642    5586 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:18.654717 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.654699    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 21:34:18.654717 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.654703    5586 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:34:18.654859 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.654816    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:34:18.654859 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:18.654815    5586 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-104.us-west-1.compute.internal"
Oct 30 21:34:18.655423 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.655404    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h8w4\" (UniqueName: \"kubernetes.io/projected/90f49eda-5385-490c-9760-2cf569eabccf-kube-api-access-4h8w4\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.655501 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.655479    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c246b\" (UniqueName: \"kubernetes.io/projected/6ad46264-9367-4a4e-8eb9-eb0d0470a669-kube-api-access-c246b\") pod \"network-metrics-daemon-w9rmz\" (UID: \"6ad46264-9367-4a4e-8eb9-eb0d0470a669\") " pod="openshift-multus/network-metrics-daemon-w9rmz"
Oct 30 21:34:18.655501 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.655496    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5rb7\" (UniqueName: \"kubernetes.io/projected/2ccf87bf-a091-4f22-8811-02dcf621e556-kube-api-access-h5rb7\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:34:18.655605 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.655582    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvjjm\" (UniqueName: \"kubernetes.io/projected/7f0cf155-c021-465e-a08b-eb2e2aa0935e-kube-api-access-mvjjm\") pod \"monitoring-plugin-77f4f76594-btfmp\" (UID: \"7f0cf155-c021-465e-a08b-eb2e2aa0935e\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:18.655672 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.655600    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv56g\" (UniqueName: \"kubernetes.io/projected/b40cbded-e2dc-420d-b454-eec7a903571e-kube-api-access-kv56g\") pod \"openshift-state-metrics-7bd7489957-2rvck\" (UID: \"b40cbded-e2dc-420d-b454-eec7a903571e\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-2rvck"
Oct 30 21:34:18.655672 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.655608    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 21:34:18.655774 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.655694    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk97l\" (UniqueName: \"kubernetes.io/projected/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-kube-api-access-wk97l\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.656721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656109    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbc9d\" (UniqueName: \"kubernetes.io/projected/06512914-5083-4ad9-a3da-a6b916f53b6e-kube-api-access-rbc9d\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:34:18.656721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656144    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x4js\" (UniqueName: \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-kube-api-access-2x4js\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.656721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656225    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xk77\" (UniqueName: \"kubernetes.io/projected/bdceec21-dc8d-4813-9637-d9b59174f16f-kube-api-access-4xk77\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:34:18.656721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656301    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjsjw\" (UniqueName: \"kubernetes.io/projected/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-api-access-qjsjw\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:34:18.656721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656365    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s46xr\" (UniqueName: \"kubernetes.io/projected/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-kube-api-access-s46xr\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:34:18.656721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656408    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4jlz\" (UniqueName: \"kubernetes.io/projected/17bdd0ae-091c-4442-a4d4-4414e8feb7b2-kube-api-access-j4jlz\") pod \"thanos-querier-5488c7d6f4-h5p6s\" (UID: \"17bdd0ae-091c-4442-a4d4-4414e8feb7b2\") " pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:18.656721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656550    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-tls\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.656721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656560    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx4xs\" (UniqueName: \"kubernetes.io/projected/5c24d279-0f41-4c11-b197-c7c9ce7ed028-kube-api-access-fx4xs\") pod \"network-check-target-2r74t\" (UID: \"5c24d279-0f41-4c11-b197-c7c9ce7ed028\") " pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:34:18.656721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656623    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ccf87bf-a091-4f22-8811-02dcf621e556-host-slash\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:34:18.656721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656681    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2ccf87bf-a091-4f22-8811-02dcf621e556-host-slash\") pod \"iptables-alerter-wdcq5\" (UID: \"2ccf87bf-a091-4f22-8811-02dcf621e556\") " pod="openshift-network-operator/iptables-alerter-wdcq5"
Oct 30 21:34:18.656721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656686    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vplh\" (UniqueName: \"kubernetes.io/projected/e83d89de-6a2a-4950-a82a-d0705739ce81-kube-api-access-9vplh\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.656721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656691    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-system-cni-dir\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.656721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656714    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmwdl\" (UniqueName: \"kubernetes.io/projected/c5deaa23-062e-431f-bca9-ffb4b60159cd-kube-api-access-kmwdl\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656750    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-system-cni-dir\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656764    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656792    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysctl-conf\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656814    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tl6d\" (UniqueName: \"kubernetes.io/projected/3bfdbb07-013c-4ab3-bf93-05af731a7b13-kube-api-access-4tl6d\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656818    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-root\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656847    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-host\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656872    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-sys\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656881    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656897    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fde4a00d-016e-4670-a530-f61d1e193a9c-dbus\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656908    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwxx8\" (UniqueName: \"kubernetes.io/projected/76696315-4c5d-4460-a3ab-e027a13211e3-kube-api-access-rwxx8\") pod \"ingress-canary-dskw4\" (UID: \"76696315-4c5d-4460-a3ab-e027a13211e3\") " pod="openshift-ingress-canary/ingress-canary-dskw4"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656921    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-system-cni-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656923    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-root\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656949    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-systemd\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656952    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9mvp\" (UniqueName: \"kubernetes.io/projected/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-kube-api-access-p9mvp\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656957    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-sys\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656965    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysctl-conf\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656972    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-host\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656982    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgfc5\" (UniqueName: \"kubernetes.io/projected/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-kube-api-access-tgfc5\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657002    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-system-cni-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.656974    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3bfdbb07-013c-4ab3-bf93-05af731a7b13-crio-socket\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657015    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-systemd\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657023    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3bfdbb07-013c-4ab3-bf93-05af731a7b13-crio-socket\") pod \"insights-runtime-extractor-csjtg\" (UID: \"3bfdbb07-013c-4ab3-bf93-05af731a7b13\") " pod="openshift-insights/insights-runtime-extractor-csjtg"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657055    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-multus-certs\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657014    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fde4a00d-016e-4670-a530-f61d1e193a9c-dbus\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657076    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-ovn\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657094    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-cni-bin\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657096    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-multus-certs\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657111    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cnibin\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657126    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-registration-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657140    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-run\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657150    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-cnibin\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657156    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-var-lib-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657159    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-cni-bin\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657193    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-run-ovn\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657201    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-var-lib-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657220    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-registration-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657218    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-cni-bin\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657268    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-run\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.657285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657294    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysctl-d\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657333    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-cni-bin\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657395    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-cni-multus\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657407    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysctl-d\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657410    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-var-lib-kubelet\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657435    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-cni-multus\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657436    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-node-log\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657457    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-node-log\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657478    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-kubernetes\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657481    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-var-lib-kubelet\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657491    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-kubernetes\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657507    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657527    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657543    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-run-netns\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657554    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657558    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-conf-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657583    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-conf-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657586    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657586    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-lib-modules\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657598    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-run-netns\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657626    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-etc-selinux\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657647    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-lib-modules\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657652    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06512914-5083-4ad9-a3da-a6b916f53b6e-host\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657670    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-kubelet\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657677    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-etc-selinux\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657697    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-systemd-units\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657701    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-kubelet\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657710    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06512914-5083-4ad9-a3da-a6b916f53b6e-host\") pod \"node-ca-jgphd\" (UID: \"06512914-5083-4ad9-a3da-a6b916f53b6e\") " pod="openshift-image-registry/node-ca-jgphd"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657722    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-k8s-cni-cncf-io\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657738    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-systemd-units\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657748    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-socket-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657770    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-socket-dir-parent\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657804    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-socket-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657806    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-k8s-cni-cncf-io\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657811    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-socket-dir-parent\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657803    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-kubelet\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657831    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-var-lib-kubelet\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657852    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-sys\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657870    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-wtmp\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657886    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-netns\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657897    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-sys\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657902    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysconfig\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657929    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42caab53-09ce-4c8f-b7ab-5d86ae4b64ba-node-exporter-wtmp\") pod \"node-exporter-zts42\" (UID: \"42caab53-09ce-4c8f-b7ab-5d86ae4b64ba\") " pod="openshift-monitoring/node-exporter-zts42"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657928    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-etc-kubernetes\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657953    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-cnibin\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657958    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-sysconfig\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657964    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-etc-kubernetes\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657967    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-systemd\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657936    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-host-run-netns\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657996    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-slash\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.657990    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-systemd\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658011    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-cnibin\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658026    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-slash\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658045    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-hosts-file\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658062    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-log-socket\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658079    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8-hosts-file\") pod \"node-resolver-z8g8q\" (UID: \"faebf7fc-ea88-4bb8-aad6-d9ae838fe5b8\") " pod="openshift-dns/node-resolver-z8g8q"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658088    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-sys-fs\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658092    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-log-socket\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658113    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658128    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-cni-netd\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658142    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-modprobe-d\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658167    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-cni-netd\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658170    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fde4a00d-016e-4670-a530-f61d1e193a9c-kubelet-config\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658161    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658197    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658200    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-sys-fs\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658221    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-cni-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658245    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fde4a00d-016e-4670-a530-f61d1e193a9c-kubelet-config\") pod \"global-pull-secret-syncer-ztncc\" (UID: \"fde4a00d-016e-4670-a530-f61d1e193a9c\") " pod="kube-system/global-pull-secret-syncer-ztncc"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658258    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/32dd7eaa-b8a6-4de5-a13a-0640b94801e8-etc-modprobe-d\") pod \"tuned-xrkmw\" (UID: \"32dd7eaa-b8a6-4de5-a13a-0640b94801e8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrkmw"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658268    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3adefe8b-2ffa-4ee1-aad4-a658970e79aa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kx4s8\" (UID: \"3adefe8b-2ffa-4ee1-aad4-a658970e79aa\") " pod="openshift-multus/multus-additional-cni-plugins-kx4s8"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658270    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-device-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658302    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-etc-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658314    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e83d89de-6a2a-4950-a82a-d0705739ce81-device-dir\") pod \"aws-ebs-csi-driver-node-n95px\" (UID: \"e83d89de-6a2a-4950-a82a-d0705739ce81\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n95px"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658327    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-hostroot\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.658318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658335    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5deaa23-062e-431f-bca9-ffb4b60159cd-etc-openvswitch\") pod \"ovnkube-node-jfbvs\" (UID: \"c5deaa23-062e-431f-bca9-ffb4b60159cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:18.660271 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658361    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-multus-cni-dir\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.660271 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.658371    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90f49eda-5385-490c-9760-2cf569eabccf-hostroot\") pod \"multus-2m72k\" (UID: \"90f49eda-5385-490c-9760-2cf569eabccf\") " pod="openshift-multus/multus-2m72k"
Oct 30 21:34:18.661672 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.661656    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3ed2089-3110-405e-928e-dcdf13e4d4e3-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p55vc\" (UID: \"c3ed2089-3110-405e-928e-dcdf13e4d4e3\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p55vc"
Oct 30 21:34:18.661672 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.661669    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bdceec21-dc8d-4813-9637-d9b59174f16f-metrics-tls\") pod \"dns-default-5svzl\" (UID: \"bdceec21-dc8d-4813-9637-d9b59174f16f\") " pod="openshift-dns/dns-default-5svzl"
Oct 30 21:34:18.662531 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.662512    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:18.664028 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:18.664011    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:20.025277 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:20.025213    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9zq7q"
Oct 30 21:34:20.729551 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:20.729513    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:20.750909 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:20.750876    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:20.776715 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:20.776690    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5svzl"
Oct 30 21:34:21.064191 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:21.064113    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:34:22.249824 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:22.249794    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5488c7d6f4-h5p6s"
Oct 30 21:34:22.899216 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:22.899188    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:23.122392 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:23.122358    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:23.449948 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:23.449915    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:23.526516 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:23.526479    5586 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:34:23.951247 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:23.951197    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:24.105414 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:24.105379    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:24.873385 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:24.873351    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-btfmp"
Oct 30 21:34:25.711582 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:25.711543    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:25.906393 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:25.906353    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:28.177895 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:28.177851    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:28.325196 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:28.325160    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:28.555060 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:28.554952    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:29.045424 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:29.045387    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:29.194246 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:29.194205    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:29.819153 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:29.819115    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:29.968089 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:29.968051    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:37.850393 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:37.850357    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:38.982948 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:38.982906    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:34:38.983480 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:38.983404    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="prometheus" containerID="cri-o://efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d" gracePeriod=600
Oct 30 21:34:38.983480 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:38.983449    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="thanos-sidecar" containerID="cri-o://ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231" gracePeriod=600
Oct 30 21:34:38.983480 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:38.983450    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="kube-rbac-proxy-web" containerID="cri-o://0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8" gracePeriod=600
Oct 30 21:34:38.983644 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:38.983478    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="config-reloader" containerID="cri-o://c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296" gracePeriod=600
Oct 30 21:34:38.983644 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:38.983478    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="kube-rbac-proxy-thanos" containerID="cri-o://a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284" gracePeriod=600
Oct 30 21:34:38.983644 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:38.983411    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="kube-rbac-proxy" containerID="cri-o://53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee" gracePeriod=600
Oct 30 21:34:39.230212 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.230183    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.329213 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329117    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-1\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329213 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329168    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-trusted-ca-bundle\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329450 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329218    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-metrics-client-certs\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329450 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329262    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-2\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329450 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329298    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-db\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329450 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329324    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-0\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329450 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329348    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-kube-rbac-proxy\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329450 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329383    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-metrics-client-ca\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329450 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329409    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-web-config\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-web-config\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329454    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-config\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-config\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329487    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-kube-api-access-2x4js\" (UniqueName: \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-kube-api-access-2x4js\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329516    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-thanos-prometheus-http-client-file\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329551    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329562    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-1" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:39.329688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329579    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-tls\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329617    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-grpc-tls\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.329688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329666    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-trusted-ca-bundle" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:39.329688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329679    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-config-out\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.330000 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329712    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-serving-certs-ca-bundle\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.330000 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329747    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-tls-assets\" (UniqueName: \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-tls-assets\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.330000 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329782    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-kubelet-serving-ca-bundle\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.330000 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.329812    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"d0534933-54d7-47c4-9302-523383fa1fc1\" (UID: \"d0534933-54d7-47c4-9302-523383fa1fc1\") "
Oct 30 21:34:39.330184 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.330002    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-metrics-client-ca" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:39.330184 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.330049    5586 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.330184 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.330067    5586 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.330919 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.330617    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-2" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:39.330919 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.330815    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-db" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:39.331108 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.330985    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-0" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:39.332630 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.332598    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-kubelet-serving-ca-bundle" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:39.334295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.334267    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-grpc-tls" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:39.334295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.332845    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-metrics-client-certs" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:39.334295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.333306    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-serving-certs-ca-bundle" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:39.334494 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.333743    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-config" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:39.334494 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.334094    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-kube-rbac-proxy" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:39.334494 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.334200    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-tls-assets" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:39.334494 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.334338    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-kube-api-access-2x4js" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "kube-api-access-2x4js". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:39.334494 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.334348    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-kube-rbac-proxy-web" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:39.334751 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.334716    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-tls" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:39.335028 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.335001    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-config-out" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:39.335209 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.335180    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-thanos-sidecar-tls" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:39.335441 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.335335    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-thanos-prometheus-http-client-file" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:39.343802 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.343777    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-web-config" pod "d0534933-54d7-47c4-9302-523383fa1fc1" (UID: "d0534933-54d7-47c4-9302-523383fa1fc1"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:39.430999 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.430951    5586 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.430999 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.430988    5586 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-kube-rbac-proxy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.430999 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.430999    5586 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-metrics-client-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431032    5586 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-web-config\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431041    5586 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-config\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431051    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x4js\" (UniqueName: \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-kube-api-access-2x4js\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431061    5586 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431070    5586 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431079    5586 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431088    5586 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-grpc-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431098    5586 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-config-out\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431107    5586 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431116    5586 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0534933-54d7-47c4-9302-523383fa1fc1-tls-assets\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431124    5586 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431134    5586 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431144    5586 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0534933-54d7-47c4-9302-523383fa1fc1-secret-metrics-client-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431153    5586 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.431281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.431161    5586 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0534933-54d7-47c4-9302-523383fa1fc1-prometheus-k8s-db\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.585180 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585089    5586 generic.go:356] "Generic (PLEG): container finished" podID="d0534933-54d7-47c4-9302-523383fa1fc1" containerID="a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284" exitCode=0
Oct 30 21:34:39.585180 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585117    5586 generic.go:356] "Generic (PLEG): container finished" podID="d0534933-54d7-47c4-9302-523383fa1fc1" containerID="53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee" exitCode=0
Oct 30 21:34:39.585180 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585129    5586 generic.go:356] "Generic (PLEG): container finished" podID="d0534933-54d7-47c4-9302-523383fa1fc1" containerID="0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8" exitCode=0
Oct 30 21:34:39.585180 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585135    5586 generic.go:356] "Generic (PLEG): container finished" podID="d0534933-54d7-47c4-9302-523383fa1fc1" containerID="ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231" exitCode=0
Oct 30 21:34:39.585180 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585142    5586 generic.go:356] "Generic (PLEG): container finished" podID="d0534933-54d7-47c4-9302-523383fa1fc1" containerID="c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296" exitCode=0
Oct 30 21:34:39.585180 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585149    5586 generic.go:356] "Generic (PLEG): container finished" podID="d0534933-54d7-47c4-9302-523383fa1fc1" containerID="efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d" exitCode=0
Oct 30 21:34:39.585180 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585168    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerDied","Data":"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"}
Oct 30 21:34:39.585655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585184    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.585655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585283    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerDied","Data":"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"}
Oct 30 21:34:39.585655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585329    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerDied","Data":"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"}
Oct 30 21:34:39.585655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585345    5586 scope.go:122] "RemoveContainer" containerID="a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"
Oct 30 21:34:39.585655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585367    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerDied","Data":"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"}
Oct 30 21:34:39.585655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585406    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerDied","Data":"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"}
Oct 30 21:34:39.585655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585448    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerDied","Data":"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"}
Oct 30 21:34:39.585655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.585489    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0534933-54d7-47c4-9302-523383fa1fc1","Type":"ContainerDied","Data":"d74eae5afc9849a57bed32c7be5be8661ec51221d568f49792c77979844689ab"}
Oct 30 21:34:39.593168 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.593149    5586 scope.go:122] "RemoveContainer" containerID="53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"
Oct 30 21:34:39.600151 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.600127    5586 scope.go:122] "RemoveContainer" containerID="0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"
Oct 30 21:34:39.607098 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.607075    5586 scope.go:122] "RemoveContainer" containerID="ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"
Oct 30 21:34:39.613789 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.613770    5586 scope.go:122] "RemoveContainer" containerID="c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"
Oct 30 21:34:39.620920 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.620890    5586 scope.go:122] "RemoveContainer" containerID="efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"
Oct 30 21:34:39.628300 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.628278    5586 scope.go:122] "RemoveContainer" containerID="76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"
Oct 30 21:34:39.635400 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.635380    5586 scope.go:122] "RemoveContainer" containerID="a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"
Oct 30 21:34:39.635725 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:39.635703    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284\": container with ID starting with a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284 not found: ID does not exist" containerID="a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"
Oct 30 21:34:39.635797 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.635734    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"} err="failed to get container status \"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284\": rpc error: code = NotFound desc = could not find container \"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284\": container with ID starting with a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284 not found: ID does not exist"
Oct 30 21:34:39.635797 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.635768    5586 scope.go:122] "RemoveContainer" containerID="53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"
Oct 30 21:34:39.636042 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:39.636020    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee\": container with ID starting with 53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee not found: ID does not exist" containerID="53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"
Oct 30 21:34:39.636092 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.636050    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"} err="failed to get container status \"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee\": rpc error: code = NotFound desc = could not find container \"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee\": container with ID starting with 53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee not found: ID does not exist"
Oct 30 21:34:39.636092 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.636068    5586 scope.go:122] "RemoveContainer" containerID="0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"
Oct 30 21:34:39.636412 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:39.636395    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8\": container with ID starting with 0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8 not found: ID does not exist" containerID="0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"
Oct 30 21:34:39.636460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.636418    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"} err="failed to get container status \"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8\": rpc error: code = NotFound desc = could not find container \"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8\": container with ID starting with 0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8 not found: ID does not exist"
Oct 30 21:34:39.636460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.636431    5586 scope.go:122] "RemoveContainer" containerID="ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"
Oct 30 21:34:39.636702 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:39.636672    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231\": container with ID starting with ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231 not found: ID does not exist" containerID="ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"
Oct 30 21:34:39.636788 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.636698    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"} err="failed to get container status \"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231\": rpc error: code = NotFound desc = could not find container \"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231\": container with ID starting with ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231 not found: ID does not exist"
Oct 30 21:34:39.636788 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.636717    5586 scope.go:122] "RemoveContainer" containerID="c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"
Oct 30 21:34:39.636990 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:39.636967    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296\": container with ID starting with c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296 not found: ID does not exist" containerID="c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"
Oct 30 21:34:39.637035 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.636997    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"} err="failed to get container status \"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296\": rpc error: code = NotFound desc = could not find container \"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296\": container with ID starting with c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296 not found: ID does not exist"
Oct 30 21:34:39.637035 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.637015    5586 scope.go:122] "RemoveContainer" containerID="efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"
Oct 30 21:34:39.637288 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:39.637270    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d\": container with ID starting with efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d not found: ID does not exist" containerID="efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"
Oct 30 21:34:39.637406 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.637296    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"} err="failed to get container status \"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d\": rpc error: code = NotFound desc = could not find container \"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d\": container with ID starting with efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d not found: ID does not exist"
Oct 30 21:34:39.637406 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.637347    5586 scope.go:122] "RemoveContainer" containerID="76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"
Oct 30 21:34:39.637608 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:39.637593    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49\": container with ID starting with 76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49 not found: ID does not exist" containerID="76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"
Oct 30 21:34:39.637644 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.637613    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"} err="failed to get container status \"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49\": rpc error: code = NotFound desc = could not find container \"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49\": container with ID starting with 76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49 not found: ID does not exist"
Oct 30 21:34:39.637644 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.637634    5586 scope.go:122] "RemoveContainer" containerID="a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"
Oct 30 21:34:39.637871 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.637849    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"} err="failed to get container status \"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284\": rpc error: code = NotFound desc = could not find container \"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284\": container with ID starting with a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284 not found: ID does not exist"
Oct 30 21:34:39.637871 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.637868    5586 scope.go:122] "RemoveContainer" containerID="53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"
Oct 30 21:34:39.638108 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.638092    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"} err="failed to get container status \"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee\": rpc error: code = NotFound desc = could not find container \"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee\": container with ID starting with 53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee not found: ID does not exist"
Oct 30 21:34:39.638108 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.638108    5586 scope.go:122] "RemoveContainer" containerID="0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"
Oct 30 21:34:39.638363 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.638346    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"} err="failed to get container status \"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8\": rpc error: code = NotFound desc = could not find container \"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8\": container with ID starting with 0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8 not found: ID does not exist"
Oct 30 21:34:39.638435 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.638364    5586 scope.go:122] "RemoveContainer" containerID="ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"
Oct 30 21:34:39.638640 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.638622    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"} err="failed to get container status \"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231\": rpc error: code = NotFound desc = could not find container \"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231\": container with ID starting with ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231 not found: ID does not exist"
Oct 30 21:34:39.638691 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.638642    5586 scope.go:122] "RemoveContainer" containerID="c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"
Oct 30 21:34:39.638868 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.638848    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"} err="failed to get container status \"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296\": rpc error: code = NotFound desc = could not find container \"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296\": container with ID starting with c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296 not found: ID does not exist"
Oct 30 21:34:39.638868 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.638867    5586 scope.go:122] "RemoveContainer" containerID="efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"
Oct 30 21:34:39.639106 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.639086    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"} err="failed to get container status \"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d\": rpc error: code = NotFound desc = could not find container \"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d\": container with ID starting with efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d not found: ID does not exist"
Oct 30 21:34:39.639147 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.639107    5586 scope.go:122] "RemoveContainer" containerID="76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"
Oct 30 21:34:39.639345 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.639326    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"} err="failed to get container status \"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49\": rpc error: code = NotFound desc = could not find container \"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49\": container with ID starting with 76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49 not found: ID does not exist"
Oct 30 21:34:39.639345 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.639345    5586 scope.go:122] "RemoveContainer" containerID="a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"
Oct 30 21:34:39.639570 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.639547    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"} err="failed to get container status \"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284\": rpc error: code = NotFound desc = could not find container \"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284\": container with ID starting with a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284 not found: ID does not exist"
Oct 30 21:34:39.639570 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.639564    5586 scope.go:122] "RemoveContainer" containerID="53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"
Oct 30 21:34:39.639789 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.639773    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"} err="failed to get container status \"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee\": rpc error: code = NotFound desc = could not find container \"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee\": container with ID starting with 53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee not found: ID does not exist"
Oct 30 21:34:39.639836 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.639789    5586 scope.go:122] "RemoveContainer" containerID="0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"
Oct 30 21:34:39.639993 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.639977    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"} err="failed to get container status \"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8\": rpc error: code = NotFound desc = could not find container \"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8\": container with ID starting with 0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8 not found: ID does not exist"
Oct 30 21:34:39.639993 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.639993    5586 scope.go:122] "RemoveContainer" containerID="ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"
Oct 30 21:34:39.640197 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.640172    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"} err="failed to get container status \"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231\": rpc error: code = NotFound desc = could not find container \"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231\": container with ID starting with ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231 not found: ID does not exist"
Oct 30 21:34:39.640197 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.640188    5586 scope.go:122] "RemoveContainer" containerID="c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"
Oct 30 21:34:39.640463 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.640447    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"} err="failed to get container status \"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296\": rpc error: code = NotFound desc = could not find container \"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296\": container with ID starting with c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296 not found: ID does not exist"
Oct 30 21:34:39.640463 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.640461    5586 scope.go:122] "RemoveContainer" containerID="efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"
Oct 30 21:34:39.640700 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.640682    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"} err="failed to get container status \"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d\": rpc error: code = NotFound desc = could not find container \"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d\": container with ID starting with efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d not found: ID does not exist"
Oct 30 21:34:39.640741 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.640701    5586 scope.go:122] "RemoveContainer" containerID="76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"
Oct 30 21:34:39.640913 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.640895    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"} err="failed to get container status \"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49\": rpc error: code = NotFound desc = could not find container \"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49\": container with ID starting with 76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49 not found: ID does not exist"
Oct 30 21:34:39.640957 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.640919    5586 scope.go:122] "RemoveContainer" containerID="a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"
Oct 30 21:34:39.641122 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.641101    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"} err="failed to get container status \"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284\": rpc error: code = NotFound desc = could not find container \"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284\": container with ID starting with a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284 not found: ID does not exist"
Oct 30 21:34:39.641161 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.641122    5586 scope.go:122] "RemoveContainer" containerID="53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"
Oct 30 21:34:39.641325 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.641309    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"} err="failed to get container status \"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee\": rpc error: code = NotFound desc = could not find container \"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee\": container with ID starting with 53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee not found: ID does not exist"
Oct 30 21:34:39.641369 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.641324    5586 scope.go:122] "RemoveContainer" containerID="0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"
Oct 30 21:34:39.641517 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.641501    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"} err="failed to get container status \"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8\": rpc error: code = NotFound desc = could not find container \"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8\": container with ID starting with 0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8 not found: ID does not exist"
Oct 30 21:34:39.641517 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.641516    5586 scope.go:122] "RemoveContainer" containerID="ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"
Oct 30 21:34:39.641710 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.641695    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"} err="failed to get container status \"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231\": rpc error: code = NotFound desc = could not find container \"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231\": container with ID starting with ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231 not found: ID does not exist"
Oct 30 21:34:39.641757 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.641709    5586 scope.go:122] "RemoveContainer" containerID="c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"
Oct 30 21:34:39.641896 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.641881    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"} err="failed to get container status \"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296\": rpc error: code = NotFound desc = could not find container \"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296\": container with ID starting with c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296 not found: ID does not exist"
Oct 30 21:34:39.641934 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.641896    5586 scope.go:122] "RemoveContainer" containerID="efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"
Oct 30 21:34:39.642130 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.642114    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"} err="failed to get container status \"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d\": rpc error: code = NotFound desc = could not find container \"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d\": container with ID starting with efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d not found: ID does not exist"
Oct 30 21:34:39.642170 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.642130    5586 scope.go:122] "RemoveContainer" containerID="76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"
Oct 30 21:34:39.642346 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.642331    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"} err="failed to get container status \"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49\": rpc error: code = NotFound desc = could not find container \"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49\": container with ID starting with 76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49 not found: ID does not exist"
Oct 30 21:34:39.642400 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.642346    5586 scope.go:122] "RemoveContainer" containerID="a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"
Oct 30 21:34:39.642574 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.642549    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"} err="failed to get container status \"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284\": rpc error: code = NotFound desc = could not find container \"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284\": container with ID starting with a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284 not found: ID does not exist"
Oct 30 21:34:39.642630 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.642574    5586 scope.go:122] "RemoveContainer" containerID="53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"
Oct 30 21:34:39.642847 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.642826    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"} err="failed to get container status \"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee\": rpc error: code = NotFound desc = could not find container \"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee\": container with ID starting with 53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee not found: ID does not exist"
Oct 30 21:34:39.642890 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.642848    5586 scope.go:122] "RemoveContainer" containerID="0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"
Oct 30 21:34:39.643086 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.643068    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"} err="failed to get container status \"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8\": rpc error: code = NotFound desc = could not find container \"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8\": container with ID starting with 0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8 not found: ID does not exist"
Oct 30 21:34:39.643137 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.643087    5586 scope.go:122] "RemoveContainer" containerID="ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"
Oct 30 21:34:39.643395 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.643368    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"} err="failed to get container status \"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231\": rpc error: code = NotFound desc = could not find container \"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231\": container with ID starting with ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231 not found: ID does not exist"
Oct 30 21:34:39.643395 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.643394    5586 scope.go:122] "RemoveContainer" containerID="c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"
Oct 30 21:34:39.643641 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.643623    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"} err="failed to get container status \"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296\": rpc error: code = NotFound desc = could not find container \"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296\": container with ID starting with c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296 not found: ID does not exist"
Oct 30 21:34:39.643688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.643641    5586 scope.go:122] "RemoveContainer" containerID="efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"
Oct 30 21:34:39.643858 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.643840    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"} err="failed to get container status \"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d\": rpc error: code = NotFound desc = could not find container \"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d\": container with ID starting with efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d not found: ID does not exist"
Oct 30 21:34:39.643913 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.643859    5586 scope.go:122] "RemoveContainer" containerID="76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"
Oct 30 21:34:39.644059 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.644045    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"} err="failed to get container status \"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49\": rpc error: code = NotFound desc = could not find container \"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49\": container with ID starting with 76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49 not found: ID does not exist"
Oct 30 21:34:39.644107 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.644059    5586 scope.go:122] "RemoveContainer" containerID="a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"
Oct 30 21:34:39.644310 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.644287    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284"} err="failed to get container status \"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284\": rpc error: code = NotFound desc = could not find container \"a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284\": container with ID starting with a091abcb2c720fb8287deaacf969faddaa60e1d07763d19b5d4c9f0d3011d284 not found: ID does not exist"
Oct 30 21:34:39.644351 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.644311    5586 scope.go:122] "RemoveContainer" containerID="53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"
Oct 30 21:34:39.644546 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.644529    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee"} err="failed to get container status \"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee\": rpc error: code = NotFound desc = could not find container \"53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee\": container with ID starting with 53100261cbceb58fa51990e294493996001b26e2fab3d27b8ec3b7f97b223fee not found: ID does not exist"
Oct 30 21:34:39.644591 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.644547    5586 scope.go:122] "RemoveContainer" containerID="0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"
Oct 30 21:34:39.644775 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.644753    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8"} err="failed to get container status \"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8\": rpc error: code = NotFound desc = could not find container \"0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8\": container with ID starting with 0d1ec2be1bc15b3143d91c528cd5f514c2837b22fd4f5e539cf40d3a4a2437d8 not found: ID does not exist"
Oct 30 21:34:39.644775 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.644773    5586 scope.go:122] "RemoveContainer" containerID="ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"
Oct 30 21:34:39.644982 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.644967    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231"} err="failed to get container status \"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231\": rpc error: code = NotFound desc = could not find container \"ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231\": container with ID starting with ad1c2304fdc44060566c82bad34d822bff7fe3a4aad884279df8b746d157b231 not found: ID does not exist"
Oct 30 21:34:39.645032 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.644982    5586 scope.go:122] "RemoveContainer" containerID="c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"
Oct 30 21:34:39.645175 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.645159    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296"} err="failed to get container status \"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296\": rpc error: code = NotFound desc = could not find container \"c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296\": container with ID starting with c52143d58b6fe499647865ad3187baf3635a1c7f8a6bbc5f448e399d825b7296 not found: ID does not exist"
Oct 30 21:34:39.645175 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.645174    5586 scope.go:122] "RemoveContainer" containerID="efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"
Oct 30 21:34:39.645388 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.645372    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d"} err="failed to get container status \"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d\": rpc error: code = NotFound desc = could not find container \"efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d\": container with ID starting with efdd6a367b617dcf688b1c955729687c4be251c8507a8ba512f676b9cf3d467d not found: ID does not exist"
Oct 30 21:34:39.645430 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.645388    5586 scope.go:122] "RemoveContainer" containerID="76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"
Oct 30 21:34:39.645628 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.645608    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49"} err="failed to get container status \"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49\": rpc error: code = NotFound desc = could not find container \"76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49\": container with ID starting with 76a5d5950a4d77d5e57610262e831948d270087fbe0f52ac4ae6c6ff7df2fb49 not found: ID does not exist"
Oct 30 21:34:39.750205 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.750170    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:34:39.756060 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.756029    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:34:39.780328 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780294    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:34:39.780635 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780618    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="thanos-sidecar"
Oct 30 21:34:39.780714 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780638    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="thanos-sidecar"
Oct 30 21:34:39.780714 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780652    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="init-config-reloader"
Oct 30 21:34:39.780714 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780662    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="init-config-reloader"
Oct 30 21:34:39.780714 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780685    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="kube-rbac-proxy"
Oct 30 21:34:39.780714 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780693    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="kube-rbac-proxy"
Oct 30 21:34:39.780714 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780705    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="config-reloader"
Oct 30 21:34:39.780714 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780712    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="config-reloader"
Oct 30 21:34:39.781052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780728    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="kube-rbac-proxy-thanos"
Oct 30 21:34:39.781052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780735    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="kube-rbac-proxy-thanos"
Oct 30 21:34:39.781052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780760    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="prometheus"
Oct 30 21:34:39.781052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780767    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="prometheus"
Oct 30 21:34:39.781052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780780    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="kube-rbac-proxy-web"
Oct 30 21:34:39.781052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780787    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="kube-rbac-proxy-web"
Oct 30 21:34:39.781052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780851    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="kube-rbac-proxy"
Oct 30 21:34:39.781052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780865    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="config-reloader"
Oct 30 21:34:39.781052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780877    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="kube-rbac-proxy-thanos"
Oct 30 21:34:39.781052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780888    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="prometheus"
Oct 30 21:34:39.781052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780897    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="kube-rbac-proxy-web"
Oct 30 21:34:39.781052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780907    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="thanos-sidecar"
Oct 30 21:34:39.781052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.780915    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" containerName="init-config-reloader"
Oct 30 21:34:39.784960 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.784935    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.837838 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.837739    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:34:39.837838 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.837831    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 21:34:39.838085 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.837838    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 21:34:39.838085 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.837936    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 21:34:39.838085 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.838057    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 21:34:39.842460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.838811    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 21:34:39.842460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.839201    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 21:34:39.842460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.839953    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 21:34:39.842460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.840991    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 21:34:39.842460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.841063    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tvbqn\""
Oct 30 21:34:39.842460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.841346    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 21:34:39.842460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.841485    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 21:34:39.842460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.841591    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 21:34:39.842460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.841654    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 21:34:39.842460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.841692    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1gtgf37ikplq8\""
Oct 30 21:34:39.888179 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.888139    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 21:34:39.888345 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.888243    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 21:34:39.888928 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.888909    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:34:39.934887 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.934851    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzj24\" (UniqueName: \"kubernetes.io/projected/76afe9bb-fda3-4e8d-a83e-a06457b7071d-kube-api-access-vzj24\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.934887 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.934889    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-config\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935096 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.934907    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/76afe9bb-fda3-4e8d-a83e-a06457b7071d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935096 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.934977    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935096 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935048    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935096 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935069    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935096 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935090    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935146    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/76afe9bb-fda3-4e8d-a83e-a06457b7071d-config-out\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935182    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935203    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935227    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935404 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935326    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935404 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935342    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935404 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935378    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935490 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935404    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935490 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935436    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935490 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935480    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935580 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935500    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-web-config\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935580 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935527    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:39.935580 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:39.935544    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.036321 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.036285    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.036780 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.036330    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.036780 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.036446    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.036780 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.036485    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-web-config\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.036780 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.036520    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.036780 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.036547    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.036780 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.036578    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzj24\" (UniqueName: \"kubernetes.io/projected/76afe9bb-fda3-4e8d-a83e-a06457b7071d-kube-api-access-vzj24\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.036780 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.036739    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-config\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.036780 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.036776    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/76afe9bb-fda3-4e8d-a83e-a06457b7071d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.037195 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.036785    5586 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:34:40.037195 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.036811    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.037195 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.036859    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.037195 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.036889    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.037195 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.036911    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.037470 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.037429    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.037637 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.037617    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.037717 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.037632    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.037777 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.037717    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.037994 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.037970    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.038075 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.038022    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/76afe9bb-fda3-4e8d-a83e-a06457b7071d-config-out\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.038075 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.038054    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.038181 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.038085    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.038181 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.038119    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.038181 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.038148    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.038181 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.038171    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.038419 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.038172    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.038419 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.038206    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.040058 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.039698    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.040058 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.040018    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/76afe9bb-fda3-4e8d-a83e-a06457b7071d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.040352 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.040326    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.040425 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.040333    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-config\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.040476 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.040432    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.041615 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.041570    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.041721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.041675    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/76afe9bb-fda3-4e8d-a83e-a06457b7071d-config-out\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.042261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.042219    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.042377 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.042351    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-web-config\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.042443 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.042418    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/76afe9bb-fda3-4e8d-a83e-a06457b7071d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.042721 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.042703    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.042801 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.042704    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.042897 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.042881    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/76afe9bb-fda3-4e8d-a83e-a06457b7071d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.093968 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.093899    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzj24\" (UniqueName: \"kubernetes.io/projected/76afe9bb-fda3-4e8d-a83e-a06457b7071d-kube-api-access-vzj24\") pod \"prometheus-k8s-0\" (UID: \"76afe9bb-fda3-4e8d-a83e-a06457b7071d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.100776 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.100748    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:40.528136 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.528079    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:34:40.532551 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:34:40.532517    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76afe9bb_fda3_4e8d_a83e_a06457b7071d.slice/crio-57653ee61bff7f4ad7e140a78d7a162a149d13af0e1618c9015e4adf0d514861 WatchSource:0}: Error finding container 57653ee61bff7f4ad7e140a78d7a162a149d13af0e1618c9015e4adf0d514861: Status 404 returned error can't find the container with id 57653ee61bff7f4ad7e140a78d7a162a149d13af0e1618c9015e4adf0d514861
Oct 30 21:34:40.590501 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:40.590469    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76afe9bb-fda3-4e8d-a83e-a06457b7071d","Type":"ContainerStarted","Data":"57653ee61bff7f4ad7e140a78d7a162a149d13af0e1618c9015e4adf0d514861"}
Oct 30 21:34:41.445497 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:41.445464    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0534933-54d7-47c4-9302-523383fa1fc1" path="/var/lib/kubelet/pods/d0534933-54d7-47c4-9302-523383fa1fc1/volumes"
Oct 30 21:34:41.596191 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:41.596156    5586 generic.go:356] "Generic (PLEG): container finished" podID="76afe9bb-fda3-4e8d-a83e-a06457b7071d" containerID="a3c7a2603225d497d222f8b91b74104189a177b9294cf19646ca1cac9e6f8756" exitCode=0
Oct 30 21:34:41.596355 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:41.596213    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76afe9bb-fda3-4e8d-a83e-a06457b7071d","Type":"ContainerDied","Data":"a3c7a2603225d497d222f8b91b74104189a177b9294cf19646ca1cac9e6f8756"}
Oct 30 21:34:42.603197 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:42.603161    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76afe9bb-fda3-4e8d-a83e-a06457b7071d","Type":"ContainerStarted","Data":"967c55f6cbc03ff2377ebad50e853262dc669a89ce0684e7155b6b7964250024"}
Oct 30 21:34:42.603588 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:42.603282    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76afe9bb-fda3-4e8d-a83e-a06457b7071d","Type":"ContainerStarted","Data":"3a8969814ed9d120df277287f40e9d190d7fe3aebcbc5453ef3b327913476a0a"}
Oct 30 21:34:42.603588 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:42.603317    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76afe9bb-fda3-4e8d-a83e-a06457b7071d","Type":"ContainerStarted","Data":"452ec8eaf52aae3fe58aaaf8922098bc7a1c2349a392f66e7cf4824ffd2d38d2"}
Oct 30 21:34:42.603588 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:42.603350    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76afe9bb-fda3-4e8d-a83e-a06457b7071d","Type":"ContainerStarted","Data":"b2cf129b593c499e55ef7a57e0a01bf4ca1d4f5e4cec215f4b704aea071f813d"}
Oct 30 21:34:42.603588 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:42.603376    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76afe9bb-fda3-4e8d-a83e-a06457b7071d","Type":"ContainerStarted","Data":"9ca3c2164bf4a6c7eb84d8dc2d9315fe7b70b20ac6beaac225d9a98c87d21aaa"}
Oct 30 21:34:42.603588 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:42.603411    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"76afe9bb-fda3-4e8d-a83e-a06457b7071d","Type":"ContainerStarted","Data":"b78a02a211485e03d29673e19d269eb4537bd8c58340820c15911726b5640952"}
Oct 30 21:34:42.727947 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:42.727875    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.710649729 podStartE2EDuration="3.727857425s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.207696ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:42.727106099 +0000 UTC m=+25.399010034" watchObservedRunningTime="2026-10-30 21:34:42.727857425 +0000 UTC m=+25.399761357"
Oct 30 21:34:44.045538 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:44.045503    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:44.097283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:44.097248    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfbvs"
Oct 30 21:34:44.246251 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:44.246185    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:45.101457 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:45.101416    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:46.468864 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:46.468827    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:46.484484 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:46.484450    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:46.633928 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:46.633895    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:34:48.361055 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:34:48.361009    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:34:51.066618 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:34:51.066582    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2r74t"
Oct 30 21:35:15.472535 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:35:15.472492    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:18.359942 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:35:18.359905    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:35:48.360141 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:35:48.360103    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:36:13.632565 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:36:13.632476    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:15.815875 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:36:15.815840    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:18.360738 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:36:18.360696    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:36:48.360176 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:36:48.360135    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:36:59.448566 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:36:59.448519    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:00.999080 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:00.999043    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:16.154406 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:16.154361    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:18.359628 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:37:18.359590    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:37:21.694546 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:21.694499    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:21.846018 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:21.845977    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:24.620970 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:24.620937    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:24.774691 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:24.774653    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:34.518892 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:34.518811    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:34.714576 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:34.714519    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:35.968024 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:35.967984    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:36.145420 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:36.145375    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:37.130639 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.130605    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:37.639982 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.639951    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"]
Oct 30 21:37:37.642267 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.642251    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"
Oct 30 21:37:37.744014 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.743973    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 21:37:37.744200 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.744041    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 21:37:37.744200 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.744070    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-2wspt\""
Oct 30 21:37:37.744611 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.744592    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"]
Oct 30 21:37:37.749420 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.749405    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7spm6\" (UniqueName: \"kubernetes.io/projected/60f75b95-423a-4894-bd28-4e5262e5ca3c-kube-api-access-7spm6\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b\" (UID: \"60f75b95-423a-4894-bd28-4e5262e5ca3c\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"
Oct 30 21:37:37.749483 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.749471    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60f75b95-423a-4894-bd28-4e5262e5ca3c-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b\" (UID: \"60f75b95-423a-4894-bd28-4e5262e5ca3c\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"
Oct 30 21:37:37.749555 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.749537    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60f75b95-423a-4894-bd28-4e5262e5ca3c-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b\" (UID: \"60f75b95-423a-4894-bd28-4e5262e5ca3c\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"
Oct 30 21:37:37.850184 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.850145    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7spm6\" (UniqueName: \"kubernetes.io/projected/60f75b95-423a-4894-bd28-4e5262e5ca3c-kube-api-access-7spm6\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b\" (UID: \"60f75b95-423a-4894-bd28-4e5262e5ca3c\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"
Oct 30 21:37:37.850406 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.850225    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60f75b95-423a-4894-bd28-4e5262e5ca3c-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b\" (UID: \"60f75b95-423a-4894-bd28-4e5262e5ca3c\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"
Oct 30 21:37:37.850406 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.850258    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60f75b95-423a-4894-bd28-4e5262e5ca3c-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b\" (UID: \"60f75b95-423a-4894-bd28-4e5262e5ca3c\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"
Oct 30 21:37:37.850645 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.850621    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60f75b95-423a-4894-bd28-4e5262e5ca3c-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b\" (UID: \"60f75b95-423a-4894-bd28-4e5262e5ca3c\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"
Oct 30 21:37:37.850725 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.850666    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60f75b95-423a-4894-bd28-4e5262e5ca3c-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b\" (UID: \"60f75b95-423a-4894-bd28-4e5262e5ca3c\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"
Oct 30 21:37:37.912815 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.912777    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7spm6\" (UniqueName: \"kubernetes.io/projected/60f75b95-423a-4894-bd28-4e5262e5ca3c-kube-api-access-7spm6\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b\" (UID: \"60f75b95-423a-4894-bd28-4e5262e5ca3c\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"
Oct 30 21:37:37.953758 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:37.953721    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"
Oct 30 21:37:38.329618 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:38.329539    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"]
Oct 30 21:37:38.333297 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:37:38.333266    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60f75b95_423a_4894_bd28_4e5262e5ca3c.slice/crio-c15f80e5edb04f80576132dc2ba792bf566da256abe5eabb9d5bed708f7f1156 WatchSource:0}: Error finding container c15f80e5edb04f80576132dc2ba792bf566da256abe5eabb9d5bed708f7f1156: Status 404 returned error can't find the container with id c15f80e5edb04f80576132dc2ba792bf566da256abe5eabb9d5bed708f7f1156
Oct 30 21:37:38.335205 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:38.335188    5586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:37:39.224877 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:39.224834    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b" event={"ID":"60f75b95-423a-4894-bd28-4e5262e5ca3c","Type":"ContainerStarted","Data":"c15f80e5edb04f80576132dc2ba792bf566da256abe5eabb9d5bed708f7f1156"}
Oct 30 21:37:40.710006 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.709971    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"]
Oct 30 21:37:40.712660 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.712637    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"
Oct 30 21:37:40.820001 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.819965    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-8dk56\""
Oct 30 21:37:40.820628 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.820601    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"]
Oct 30 21:37:40.873895 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.873857    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5522007-4832-4a97-bef2-41e82acd0553-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x\" (UID: \"a5522007-4832-4a97-bef2-41e82acd0553\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"
Oct 30 21:37:40.874088 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.873916    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5522007-4832-4a97-bef2-41e82acd0553-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x\" (UID: \"a5522007-4832-4a97-bef2-41e82acd0553\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"
Oct 30 21:37:40.874088 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.874070    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzhbb\" (UniqueName: \"kubernetes.io/projected/a5522007-4832-4a97-bef2-41e82acd0553-kube-api-access-lzhbb\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x\" (UID: \"a5522007-4832-4a97-bef2-41e82acd0553\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"
Oct 30 21:37:40.902154 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.902113    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"]
Oct 30 21:37:40.904332 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.904307    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"
Oct 30 21:37:40.960957 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.960873    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-9c8sj\""
Oct 30 21:37:40.974050 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.974017    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"]
Oct 30 21:37:40.974736 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.974702    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzhbb\" (UniqueName: \"kubernetes.io/projected/a5522007-4832-4a97-bef2-41e82acd0553-kube-api-access-lzhbb\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x\" (UID: \"a5522007-4832-4a97-bef2-41e82acd0553\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"
Oct 30 21:37:40.974846 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.974767    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5522007-4832-4a97-bef2-41e82acd0553-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x\" (UID: \"a5522007-4832-4a97-bef2-41e82acd0553\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"
Oct 30 21:37:40.974846 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.974804    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5522007-4832-4a97-bef2-41e82acd0553-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x\" (UID: \"a5522007-4832-4a97-bef2-41e82acd0553\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"
Oct 30 21:37:40.975212 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.975194    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5522007-4832-4a97-bef2-41e82acd0553-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x\" (UID: \"a5522007-4832-4a97-bef2-41e82acd0553\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"
Oct 30 21:37:40.975212 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:40.975204    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5522007-4832-4a97-bef2-41e82acd0553-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x\" (UID: \"a5522007-4832-4a97-bef2-41e82acd0553\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"
Oct 30 21:37:41.035717 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.035687    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzhbb\" (UniqueName: \"kubernetes.io/projected/a5522007-4832-4a97-bef2-41e82acd0553-kube-api-access-lzhbb\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x\" (UID: \"a5522007-4832-4a97-bef2-41e82acd0553\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"
Oct 30 21:37:41.051189 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.051161    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"]
Oct 30 21:37:41.053807 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.053788    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"
Oct 30 21:37:41.075189 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.075161    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4czp\" (UniqueName: \"kubernetes.io/projected/a9b5ccfb-59ae-463c-b93e-e60b632538e0-kube-api-access-x4czp\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf\" (UID: \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"
Oct 30 21:37:41.075345 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.075204    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9b5ccfb-59ae-463c-b93e-e60b632538e0-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf\" (UID: \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"
Oct 30 21:37:41.075397 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.075377    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9b5ccfb-59ae-463c-b93e-e60b632538e0-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf\" (UID: \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"
Oct 30 21:37:41.106749 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.106712    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-5tsl7\""
Oct 30 21:37:41.113831 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.113796    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"]
Oct 30 21:37:41.146682 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.146647    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"]
Oct 30 21:37:41.149300 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.149277    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"
Oct 30 21:37:41.176202 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.176174    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4czp\" (UniqueName: \"kubernetes.io/projected/a9b5ccfb-59ae-463c-b93e-e60b632538e0-kube-api-access-x4czp\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf\" (UID: \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"
Oct 30 21:37:41.176387 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.176208    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79722d5b-75c0-4443-98ea-dd16ee5a841b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp\" (UID: \"79722d5b-75c0-4443-98ea-dd16ee5a841b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"
Oct 30 21:37:41.176387 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.176254    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9b5ccfb-59ae-463c-b93e-e60b632538e0-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf\" (UID: \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"
Oct 30 21:37:41.176387 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.176284    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79722d5b-75c0-4443-98ea-dd16ee5a841b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp\" (UID: \"79722d5b-75c0-4443-98ea-dd16ee5a841b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"
Oct 30 21:37:41.176387 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.176325    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9b5ccfb-59ae-463c-b93e-e60b632538e0-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf\" (UID: \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"
Oct 30 21:37:41.176387 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.176360    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4dlk\" (UniqueName: \"kubernetes.io/projected/79722d5b-75c0-4443-98ea-dd16ee5a841b-kube-api-access-r4dlk\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp\" (UID: \"79722d5b-75c0-4443-98ea-dd16ee5a841b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"
Oct 30 21:37:41.177022 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.176995    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9b5ccfb-59ae-463c-b93e-e60b632538e0-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf\" (UID: \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"
Oct 30 21:37:41.177310 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.177291    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9b5ccfb-59ae-463c-b93e-e60b632538e0-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf\" (UID: \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"
Oct 30 21:37:41.202005 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.201974    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-8k2sm\""
Oct 30 21:37:41.209401 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.209374    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"]
Oct 30 21:37:41.234658 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.234589    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4czp\" (UniqueName: \"kubernetes.io/projected/a9b5ccfb-59ae-463c-b93e-e60b632538e0-kube-api-access-x4czp\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf\" (UID: \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"
Oct 30 21:37:41.277614 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.277577    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79722d5b-75c0-4443-98ea-dd16ee5a841b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp\" (UID: \"79722d5b-75c0-4443-98ea-dd16ee5a841b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"
Oct 30 21:37:41.277796 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.277625    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d36b478-37a8-4290-bc21-dc034fb169fa-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz\" (UID: \"9d36b478-37a8-4290-bc21-dc034fb169fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"
Oct 30 21:37:41.277796 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.277651    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4dlk\" (UniqueName: \"kubernetes.io/projected/79722d5b-75c0-4443-98ea-dd16ee5a841b-kube-api-access-r4dlk\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp\" (UID: \"79722d5b-75c0-4443-98ea-dd16ee5a841b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"
Oct 30 21:37:41.277796 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.277693    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d36b478-37a8-4290-bc21-dc034fb169fa-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz\" (UID: \"9d36b478-37a8-4290-bc21-dc034fb169fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"
Oct 30 21:37:41.277796 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.277769    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz8nm\" (UniqueName: \"kubernetes.io/projected/9d36b478-37a8-4290-bc21-dc034fb169fa-kube-api-access-zz8nm\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz\" (UID: \"9d36b478-37a8-4290-bc21-dc034fb169fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"
Oct 30 21:37:41.277997 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.277869    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79722d5b-75c0-4443-98ea-dd16ee5a841b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp\" (UID: \"79722d5b-75c0-4443-98ea-dd16ee5a841b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"
Oct 30 21:37:41.278050 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.278026    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79722d5b-75c0-4443-98ea-dd16ee5a841b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp\" (UID: \"79722d5b-75c0-4443-98ea-dd16ee5a841b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"
Oct 30 21:37:41.278189 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.278170    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79722d5b-75c0-4443-98ea-dd16ee5a841b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp\" (UID: \"79722d5b-75c0-4443-98ea-dd16ee5a841b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"
Oct 30 21:37:41.327402 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.327367    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"
Oct 30 21:37:41.336971 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.336944    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4dlk\" (UniqueName: \"kubernetes.io/projected/79722d5b-75c0-4443-98ea-dd16ee5a841b-kube-api-access-r4dlk\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp\" (UID: \"79722d5b-75c0-4443-98ea-dd16ee5a841b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"
Oct 30 21:37:41.368578 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.368543    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"
Oct 30 21:37:41.379330 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.379303    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d36b478-37a8-4290-bc21-dc034fb169fa-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz\" (UID: \"9d36b478-37a8-4290-bc21-dc034fb169fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"
Oct 30 21:37:41.379463 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.379345    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz8nm\" (UniqueName: \"kubernetes.io/projected/9d36b478-37a8-4290-bc21-dc034fb169fa-kube-api-access-zz8nm\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz\" (UID: \"9d36b478-37a8-4290-bc21-dc034fb169fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"
Oct 30 21:37:41.379463 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.379420    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d36b478-37a8-4290-bc21-dc034fb169fa-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz\" (UID: \"9d36b478-37a8-4290-bc21-dc034fb169fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"
Oct 30 21:37:41.379873 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.379828    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d36b478-37a8-4290-bc21-dc034fb169fa-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz\" (UID: \"9d36b478-37a8-4290-bc21-dc034fb169fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"
Oct 30 21:37:41.379873 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.379842    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d36b478-37a8-4290-bc21-dc034fb169fa-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz\" (UID: \"9d36b478-37a8-4290-bc21-dc034fb169fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"
Oct 30 21:37:41.438540 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.438468    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz8nm\" (UniqueName: \"kubernetes.io/projected/9d36b478-37a8-4290-bc21-dc034fb169fa-kube-api-access-zz8nm\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz\" (UID: \"9d36b478-37a8-4290-bc21-dc034fb169fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"
Oct 30 21:37:41.463433 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.463386    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"
Oct 30 21:37:41.518683 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.518339    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"
Oct 30 21:37:41.713118 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.713084    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"]
Oct 30 21:37:41.716060 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.716039    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"
Oct 30 21:37:41.752772 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.752687    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"]
Oct 30 21:37:41.757149 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:37:41.757118    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5522007_4832_4a97_bef2_41e82acd0553.slice/crio-311f1731c2461a84ed93e1473a357824c0235acd944e0d65725b109836e6b80f WatchSource:0}: Error finding container 311f1731c2461a84ed93e1473a357824c0235acd944e0d65725b109836e6b80f: Status 404 returned error can't find the container with id 311f1731c2461a84ed93e1473a357824c0235acd944e0d65725b109836e6b80f
Oct 30 21:37:41.758144 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.758081    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"]
Oct 30 21:37:41.768726 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.768672    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-6d7kc\""
Oct 30 21:37:41.772707 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:37:41.772680    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79722d5b_75c0_4443_98ea_dd16ee5a841b.slice/crio-ac5c03d5e7ea9bcf11f81cb3eb5dc8a2494da3c14cfe2d74f300de633401263a WatchSource:0}: Error finding container ac5c03d5e7ea9bcf11f81cb3eb5dc8a2494da3c14cfe2d74f300de633401263a: Status 404 returned error can't find the container with id ac5c03d5e7ea9bcf11f81cb3eb5dc8a2494da3c14cfe2d74f300de633401263a
Oct 30 21:37:41.777733 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.777707    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"]
Oct 30 21:37:41.871899 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.871855    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"]
Oct 30 21:37:41.875934 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:37:41.875896    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d36b478_37a8_4290_bc21_dc034fb169fa.slice/crio-e7042b1891dfce0f2205003359722cccdc16c9a8f856b8f2eac183a4d5c01092 WatchSource:0}: Error finding container e7042b1891dfce0f2205003359722cccdc16c9a8f856b8f2eac183a4d5c01092: Status 404 returned error can't find the container with id e7042b1891dfce0f2205003359722cccdc16c9a8f856b8f2eac183a4d5c01092
Oct 30 21:37:41.883969 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.883940    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfwk6\" (UniqueName: \"kubernetes.io/projected/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-kube-api-access-lfwk6\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr\" (UID: \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"
Oct 30 21:37:41.884102 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.884017    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr\" (UID: \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"
Oct 30 21:37:41.884102 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.884093    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr\" (UID: \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"
Oct 30 21:37:41.907021 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.906978    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"]
Oct 30 21:37:41.912625 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:37:41.912591    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b5ccfb_59ae_463c_b93e_e60b632538e0.slice/crio-0b9eb3872d9fbe7ba0369094cdfe71d1282b7d7e39e19a5c3eac9704d87f3df5 WatchSource:0}: Error finding container 0b9eb3872d9fbe7ba0369094cdfe71d1282b7d7e39e19a5c3eac9704d87f3df5: Status 404 returned error can't find the container with id 0b9eb3872d9fbe7ba0369094cdfe71d1282b7d7e39e19a5c3eac9704d87f3df5
Oct 30 21:37:41.985099 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.985057    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr\" (UID: \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"
Oct 30 21:37:41.985300 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.985115    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfwk6\" (UniqueName: \"kubernetes.io/projected/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-kube-api-access-lfwk6\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr\" (UID: \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"
Oct 30 21:37:41.985300 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.985177    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr\" (UID: \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"
Oct 30 21:37:41.985558 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.985534    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr\" (UID: \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"
Oct 30 21:37:41.985637 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:41.985594    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr\" (UID: \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"
Oct 30 21:37:42.046972 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:42.046899    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfwk6\" (UniqueName: \"kubernetes.io/projected/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-kube-api-access-lfwk6\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr\" (UID: \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"
Oct 30 21:37:42.239146 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:42.239106    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz" event={"ID":"9d36b478-37a8-4290-bc21-dc034fb169fa","Type":"ContainerStarted","Data":"e7042b1891dfce0f2205003359722cccdc16c9a8f856b8f2eac183a4d5c01092"}
Oct 30 21:37:42.240441 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:42.240406    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp" event={"ID":"79722d5b-75c0-4443-98ea-dd16ee5a841b","Type":"ContainerStarted","Data":"ac5c03d5e7ea9bcf11f81cb3eb5dc8a2494da3c14cfe2d74f300de633401263a"}
Oct 30 21:37:42.241741 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:42.241711    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x" event={"ID":"a5522007-4832-4a97-bef2-41e82acd0553","Type":"ContainerStarted","Data":"311f1731c2461a84ed93e1473a357824c0235acd944e0d65725b109836e6b80f"}
Oct 30 21:37:42.242906 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:42.242883    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf" event={"ID":"a9b5ccfb-59ae-463c-b93e-e60b632538e0","Type":"ContainerStarted","Data":"0b9eb3872d9fbe7ba0369094cdfe71d1282b7d7e39e19a5c3eac9704d87f3df5"}
Oct 30 21:37:42.330598 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:42.330507    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"
Oct 30 21:37:42.719821 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:42.719713    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"]
Oct 30 21:37:42.733893 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:37:42.733845    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod431bf312_0bf1_41b4_9f30_c5cafe4bd0ce.slice/crio-d6e4db0c0159ccd5c4fbfda55e53d38cbda2b87b120e2f521b88208673ff3ca8 WatchSource:0}: Error finding container d6e4db0c0159ccd5c4fbfda55e53d38cbda2b87b120e2f521b88208673ff3ca8: Status 404 returned error can't find the container with id d6e4db0c0159ccd5c4fbfda55e53d38cbda2b87b120e2f521b88208673ff3ca8
Oct 30 21:37:42.944033 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:42.942192    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"]
Oct 30 21:37:42.945704 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:42.944968    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"
Oct 30 21:37:42.997907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:42.997829    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-9fs5n\""
Oct 30 21:37:43.007565 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:43.007504    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"]
Oct 30 21:37:43.096046 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:43.096006    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wp9z\" (UniqueName: \"kubernetes.io/projected/a1f9a138-94b8-490d-a1eb-32a6ece64758-kube-api-access-2wp9z\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4\" (UID: \"a1f9a138-94b8-490d-a1eb-32a6ece64758\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"
Oct 30 21:37:43.096277 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:43.096061    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1f9a138-94b8-490d-a1eb-32a6ece64758-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4\" (UID: \"a1f9a138-94b8-490d-a1eb-32a6ece64758\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"
Oct 30 21:37:43.096277 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:43.096095    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1f9a138-94b8-490d-a1eb-32a6ece64758-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4\" (UID: \"a1f9a138-94b8-490d-a1eb-32a6ece64758\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"
Oct 30 21:37:43.197397 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:43.197358    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wp9z\" (UniqueName: \"kubernetes.io/projected/a1f9a138-94b8-490d-a1eb-32a6ece64758-kube-api-access-2wp9z\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4\" (UID: \"a1f9a138-94b8-490d-a1eb-32a6ece64758\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"
Oct 30 21:37:43.197586 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:43.197433    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1f9a138-94b8-490d-a1eb-32a6ece64758-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4\" (UID: \"a1f9a138-94b8-490d-a1eb-32a6ece64758\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"
Oct 30 21:37:43.197586 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:43.197501    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1f9a138-94b8-490d-a1eb-32a6ece64758-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4\" (UID: \"a1f9a138-94b8-490d-a1eb-32a6ece64758\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"
Oct 30 21:37:43.197922 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:43.197890    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1f9a138-94b8-490d-a1eb-32a6ece64758-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4\" (UID: \"a1f9a138-94b8-490d-a1eb-32a6ece64758\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"
Oct 30 21:37:43.198434 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:43.198413    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1f9a138-94b8-490d-a1eb-32a6ece64758-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4\" (UID: \"a1f9a138-94b8-490d-a1eb-32a6ece64758\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"
Oct 30 21:37:43.251191 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:43.251098    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr" event={"ID":"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce","Type":"ContainerStarted","Data":"d6e4db0c0159ccd5c4fbfda55e53d38cbda2b87b120e2f521b88208673ff3ca8"}
Oct 30 21:37:43.261668 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:43.261598    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wp9z\" (UniqueName: \"kubernetes.io/projected/a1f9a138-94b8-490d-a1eb-32a6ece64758-kube-api-access-2wp9z\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4\" (UID: \"a1f9a138-94b8-490d-a1eb-32a6ece64758\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"
Oct 30 21:37:43.263242 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:43.263209    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"
Oct 30 21:37:43.684777 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:43.684723    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"]
Oct 30 21:37:43.690263 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:37:43.690178    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1f9a138_94b8_490d_a1eb_32a6ece64758.slice/crio-73b49fda24c54b1930aa4319e96c38b75e1d2c34e74f55981dbd97e3f6a78e63 WatchSource:0}: Error finding container 73b49fda24c54b1930aa4319e96c38b75e1d2c34e74f55981dbd97e3f6a78e63: Status 404 returned error can't find the container with id 73b49fda24c54b1930aa4319e96c38b75e1d2c34e74f55981dbd97e3f6a78e63
Oct 30 21:37:44.261785 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:44.261725    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4" event={"ID":"a1f9a138-94b8-490d-a1eb-32a6ece64758","Type":"ContainerStarted","Data":"73b49fda24c54b1930aa4319e96c38b75e1d2c34e74f55981dbd97e3f6a78e63"}
Oct 30 21:37:47.281743 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:47.281703    5586 generic.go:356] "Generic (PLEG): container finished" podID="60f75b95-423a-4894-bd28-4e5262e5ca3c" containerID="8d77a316dab4a604f762097f4447b1301b783d54c61b389029c095ceb4b37efb" exitCode=0
Oct 30 21:37:47.282247 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:47.281797    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b" event={"ID":"60f75b95-423a-4894-bd28-4e5262e5ca3c","Type":"ContainerDied","Data":"8d77a316dab4a604f762097f4447b1301b783d54c61b389029c095ceb4b37efb"}
Oct 30 21:37:48.385796 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:37:48.385724    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:37:49.297413 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:49.296461    5586 generic.go:356] "Generic (PLEG): container finished" podID="60f75b95-423a-4894-bd28-4e5262e5ca3c" containerID="cdb7aa08e98dbe0263803f7c67486969649c6fa7351a72620018bde4a9f6ef0b" exitCode=0
Oct 30 21:37:49.297413 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:49.296521    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b" event={"ID":"60f75b95-423a-4894-bd28-4e5262e5ca3c","Type":"ContainerDied","Data":"cdb7aa08e98dbe0263803f7c67486969649c6fa7351a72620018bde4a9f6ef0b"}
Oct 30 21:37:50.304961 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:50.304925    5586 generic.go:356] "Generic (PLEG): container finished" podID="9d36b478-37a8-4290-bc21-dc034fb169fa" containerID="1a978efc88b4ebbe8e6b2807b8e27466e16baa12a839e60f93005f5c0af2bea5" exitCode=0
Oct 30 21:37:50.305439 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:50.305034    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz" event={"ID":"9d36b478-37a8-4290-bc21-dc034fb169fa","Type":"ContainerDied","Data":"1a978efc88b4ebbe8e6b2807b8e27466e16baa12a839e60f93005f5c0af2bea5"}
Oct 30 21:37:50.309357 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:50.309249    5586 generic.go:356] "Generic (PLEG): container finished" podID="79722d5b-75c0-4443-98ea-dd16ee5a841b" containerID="a1216b89d4b12d8df2926b7884a4c311cef7e9c3badc9e5f819dd9f4ea554b6f" exitCode=0
Oct 30 21:37:50.309357 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:50.309327    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp" event={"ID":"79722d5b-75c0-4443-98ea-dd16ee5a841b","Type":"ContainerDied","Data":"a1216b89d4b12d8df2926b7884a4c311cef7e9c3badc9e5f819dd9f4ea554b6f"}
Oct 30 21:37:50.313698 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:50.313669    5586 generic.go:356] "Generic (PLEG): container finished" podID="a5522007-4832-4a97-bef2-41e82acd0553" containerID="6be2aa008b6fd0ed4704219317ddc07fe3c62c9e64044e4f0545dfc9e24df035" exitCode=0
Oct 30 21:37:50.313881 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:50.313840    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x" event={"ID":"a5522007-4832-4a97-bef2-41e82acd0553","Type":"ContainerDied","Data":"6be2aa008b6fd0ed4704219317ddc07fe3c62c9e64044e4f0545dfc9e24df035"}
Oct 30 21:37:50.317138 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:50.316496    5586 generic.go:356] "Generic (PLEG): container finished" podID="a9b5ccfb-59ae-463c-b93e-e60b632538e0" containerID="4fb05729e00cce18139e0efe54336cb4336e5e610fd55f08c1ebf92632b4b2bf" exitCode=0
Oct 30 21:37:50.317138 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:50.316538    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf" event={"ID":"a9b5ccfb-59ae-463c-b93e-e60b632538e0","Type":"ContainerDied","Data":"4fb05729e00cce18139e0efe54336cb4336e5e610fd55f08c1ebf92632b4b2bf"}
Oct 30 21:37:51.323654 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:51.323618    5586 generic.go:356] "Generic (PLEG): container finished" podID="431bf312-0bf1-41b4-9f30-c5cafe4bd0ce" containerID="502e44fc63203809a610ac7514743df07439533a151b33f18e5086b586debd5b" exitCode=0
Oct 30 21:37:51.324160 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:51.323712    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr" event={"ID":"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce","Type":"ContainerDied","Data":"502e44fc63203809a610ac7514743df07439533a151b33f18e5086b586debd5b"}
Oct 30 21:37:51.325803 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:51.325778    5586 generic.go:356] "Generic (PLEG): container finished" podID="a1f9a138-94b8-490d-a1eb-32a6ece64758" containerID="ac514267552ae15e53c9f0d8372c27bdb6d0a882b7a5dd296b0f23f313cd4a85" exitCode=0
Oct 30 21:37:51.325955 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:51.325902    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4" event={"ID":"a1f9a138-94b8-490d-a1eb-32a6ece64758","Type":"ContainerDied","Data":"ac514267552ae15e53c9f0d8372c27bdb6d0a882b7a5dd296b0f23f313cd4a85"}
Oct 30 21:37:51.849581 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:51.849547    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:52.332907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:52.332876    5586 generic.go:356] "Generic (PLEG): container finished" podID="79722d5b-75c0-4443-98ea-dd16ee5a841b" containerID="5484fb8d7f96f8acd2a8fd1b31d6c3753d15b7d0f3836462ffe8e02dbfecdd67" exitCode=0
Oct 30 21:37:52.333392 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:52.332979    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp" event={"ID":"79722d5b-75c0-4443-98ea-dd16ee5a841b","Type":"ContainerDied","Data":"5484fb8d7f96f8acd2a8fd1b31d6c3753d15b7d0f3836462ffe8e02dbfecdd67"}
Oct 30 21:37:52.335319 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:52.335299    5586 generic.go:356] "Generic (PLEG): container finished" podID="a5522007-4832-4a97-bef2-41e82acd0553" containerID="514e54ae8c3e18c8455ef5e91d34a18cfc0af1c082b9f712c0c843085335525a" exitCode=0
Oct 30 21:37:52.335410 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:52.335369    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x" event={"ID":"a5522007-4832-4a97-bef2-41e82acd0553","Type":"ContainerDied","Data":"514e54ae8c3e18c8455ef5e91d34a18cfc0af1c082b9f712c0c843085335525a"}
Oct 30 21:37:52.338325 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:52.338304    5586 generic.go:356] "Generic (PLEG): container finished" podID="a9b5ccfb-59ae-463c-b93e-e60b632538e0" containerID="359a327e62aad5185a0867826c2ec91c0b987800523f8b7bffce919ac5903ea5" exitCode=0
Oct 30 21:37:52.338433 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:52.338341    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf" event={"ID":"a9b5ccfb-59ae-463c-b93e-e60b632538e0","Type":"ContainerDied","Data":"359a327e62aad5185a0867826c2ec91c0b987800523f8b7bffce919ac5903ea5"}
Oct 30 21:37:52.768743 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:52.768647    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:53.175933 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.175900    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"]
Oct 30 21:37:53.180468 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.180442    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.194414 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.194333    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.194414 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.194387    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edcb968b-97be-45b0-8e26-39378e683676-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.194637 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.194499    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.194637 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.194539    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.194637 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.194583    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.194637 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.194609    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edcb968b-97be-45b0-8e26-39378e683676-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.194855 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.194644    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.194855 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.194676    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2tt9\" (UniqueName: \"kubernetes.io/projected/edcb968b-97be-45b0-8e26-39378e683676-kube-api-access-g2tt9\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.194855 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.194705    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edcb968b-97be-45b0-8e26-39378e683676-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.235585 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.235549    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 21:37:53.235771 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.235725    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 21:37:53.235861 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.235791    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 21:37:53.246700 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.246627    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-xn2np\""
Oct 30 21:37:53.288287 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.287733    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"]
Oct 30 21:37:53.296869 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.295897    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edcb968b-97be-45b0-8e26-39378e683676-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.296869 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.295954    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.296869 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.296011    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.296869 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.296052    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.296869 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.296148    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edcb968b-97be-45b0-8e26-39378e683676-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.296869 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.296213    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.296869 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.296267    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2tt9\" (UniqueName: \"kubernetes.io/projected/edcb968b-97be-45b0-8e26-39378e683676-kube-api-access-g2tt9\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.296869 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.296312    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edcb968b-97be-45b0-8e26-39378e683676-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.296869 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.296342    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.296869 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.296760    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.299348 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.299293    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.299796 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.299770    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.300018 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.299996    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.300766 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.300718    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edcb968b-97be-45b0-8e26-39378e683676-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.305075 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.305050    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:53.307439 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.307416    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.319091 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.319029    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edcb968b-97be-45b0-8e26-39378e683676-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.349736 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.348732    5586 generic.go:356] "Generic (PLEG): container finished" podID="9d36b478-37a8-4290-bc21-dc034fb169fa" containerID="fe6530a2e99abf3385535a689101ab758ba1d0d69a348b2186bc41bf5e54e3f0" exitCode=0
Oct 30 21:37:53.349736 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.348790    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz" event={"ID":"9d36b478-37a8-4290-bc21-dc034fb169fa","Type":"ContainerDied","Data":"fe6530a2e99abf3385535a689101ab758ba1d0d69a348b2186bc41bf5e54e3f0"}
Oct 30 21:37:53.356438 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.356364    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edcb968b-97be-45b0-8e26-39378e683676-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.358485 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.358458    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2tt9\" (UniqueName: \"kubernetes.io/projected/edcb968b-97be-45b0-8e26-39378e683676-kube-api-access-g2tt9\") pod \"maas-default-gateway-openshift-default-74854d8ccf-24vpd\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.480308 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.480147    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:53.499338 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.499292    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:37:53.672807 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.672685    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:53.824747 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.824652    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:53.977525 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:53.977491    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"]
Oct 30 21:37:53.994916 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:37:53.987481    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedcb968b_97be_45b0_8e26_39378e683676.slice/crio-b00f63944185b3a1b9e7f2a4b224370d76d2d40f99d6bb17e62b8db5b003faaf WatchSource:0}: Error finding container b00f63944185b3a1b9e7f2a4b224370d76d2d40f99d6bb17e62b8db5b003faaf: Status 404 returned error can't find the container with id b00f63944185b3a1b9e7f2a4b224370d76d2d40f99d6bb17e62b8db5b003faaf
Oct 30 21:37:54.362064 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:54.361992    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd" event={"ID":"edcb968b-97be-45b0-8e26-39378e683676","Type":"ContainerStarted","Data":"b00f63944185b3a1b9e7f2a4b224370d76d2d40f99d6bb17e62b8db5b003faaf"}
Oct 30 21:37:57.383433 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:57.383383    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr" event={"ID":"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce","Type":"ContainerStarted","Data":"c8a25bb179e078e32ea67a871f28be174e718801bc2a400a7d82fed2f6abf7cf"}
Oct 30 21:37:57.406560 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:57.406505    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:58.397634 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:58.397561    5586 generic.go:356] "Generic (PLEG): container finished" podID="431bf312-0bf1-41b4-9f30-c5cafe4bd0ce" containerID="c8a25bb179e078e32ea67a871f28be174e718801bc2a400a7d82fed2f6abf7cf" exitCode=0
Oct 30 21:37:58.398094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:58.397651    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr" event={"ID":"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce","Type":"ContainerDied","Data":"c8a25bb179e078e32ea67a871f28be174e718801bc2a400a7d82fed2f6abf7cf"}
Oct 30 21:37:58.405048 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:58.405017    5586 generic.go:356] "Generic (PLEG): container finished" podID="a1f9a138-94b8-490d-a1eb-32a6ece64758" containerID="35f7c0bd4433f3062fadcaec10842813da15381e292a3abd48ae0bf09fcbfa03" exitCode=0
Oct 30 21:37:58.405142 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:37:58.405126    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4" event={"ID":"a1f9a138-94b8-490d-a1eb-32a6ece64758","Type":"ContainerDied","Data":"35f7c0bd4433f3062fadcaec10842813da15381e292a3abd48ae0bf09fcbfa03"}
Oct 30 21:38:02.439816 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:02.439771    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x" event={"ID":"a5522007-4832-4a97-bef2-41e82acd0553","Type":"ContainerStarted","Data":"58cd63eb042150a64d0f7218882e758abef8e21ef436ea7e0261c6b3af77cf18"}
Oct 30 21:38:02.445835 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:02.445167    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b" event={"ID":"60f75b95-423a-4894-bd28-4e5262e5ca3c","Type":"ContainerStarted","Data":"98f502acccbc477f50b61c8e140aae3bf2a492126af542fa831727c7cd6040dd"}
Oct 30 21:38:02.448949 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:02.448881    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf" event={"ID":"a9b5ccfb-59ae-463c-b93e-e60b632538e0","Type":"ContainerStarted","Data":"436d5224d3d068832913022920f70923075c3375cccbb7c44c65bad6029e245a"}
Oct 30 21:38:02.562915 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:02.562851    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x" podStartSLOduration=2.744730192 podStartE2EDuration="22.562830275s" totalImagesPullingTime="19.818100083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:02.56263335 +0000 UTC m=+225.234537285" watchObservedRunningTime="2026-10-30 21:38:02.562830275 +0000 UTC m=+225.234734211"
Oct 30 21:38:02.678308 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:02.678227    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b" podStartSLOduration=3.599676922 podStartE2EDuration="25.678211377s" totalImagesPullingTime="22.078534455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:02.678063726 +0000 UTC m=+225.349967662" watchObservedRunningTime="2026-10-30 21:38:02.678211377 +0000 UTC m=+225.350115313"
Oct 30 21:38:02.775908 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:02.775866    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:38:02.776039 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:02.775968    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:38:02.776039 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:02.776018    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:38:03.458088 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:03.457999    5586 generic.go:356] "Generic (PLEG): container finished" podID="79722d5b-75c0-4443-98ea-dd16ee5a841b" containerID="dd7cf943afe0ea9050958638117b3798287ad2f1c57f98f24a97db92dac21207" exitCode=0
Oct 30 21:38:03.458625 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:03.458108    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp" event={"ID":"79722d5b-75c0-4443-98ea-dd16ee5a841b","Type":"ContainerDied","Data":"dd7cf943afe0ea9050958638117b3798287ad2f1c57f98f24a97db92dac21207"}
Oct 30 21:38:03.462914 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:03.462881    5586 generic.go:356] "Generic (PLEG): container finished" podID="a5522007-4832-4a97-bef2-41e82acd0553" containerID="58cd63eb042150a64d0f7218882e758abef8e21ef436ea7e0261c6b3af77cf18" exitCode=0
Oct 30 21:38:03.463051 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:03.462956    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x" event={"ID":"a5522007-4832-4a97-bef2-41e82acd0553","Type":"ContainerDied","Data":"58cd63eb042150a64d0f7218882e758abef8e21ef436ea7e0261c6b3af77cf18"}
Oct 30 21:38:03.465755 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:03.465725    5586 generic.go:356] "Generic (PLEG): container finished" podID="60f75b95-423a-4894-bd28-4e5262e5ca3c" containerID="98f502acccbc477f50b61c8e140aae3bf2a492126af542fa831727c7cd6040dd" exitCode=0
Oct 30 21:38:03.465889 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:03.465777    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b" event={"ID":"60f75b95-423a-4894-bd28-4e5262e5ca3c","Type":"ContainerDied","Data":"98f502acccbc477f50b61c8e140aae3bf2a492126af542fa831727c7cd6040dd"}
Oct 30 21:38:03.468959 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:03.468934    5586 generic.go:356] "Generic (PLEG): container finished" podID="a9b5ccfb-59ae-463c-b93e-e60b632538e0" containerID="436d5224d3d068832913022920f70923075c3375cccbb7c44c65bad6029e245a" exitCode=0
Oct 30 21:38:03.469068 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:03.469031    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf" event={"ID":"a9b5ccfb-59ae-463c-b93e-e60b632538e0","Type":"ContainerDied","Data":"436d5224d3d068832913022920f70923075c3375cccbb7c44c65bad6029e245a"}
Oct 30 21:38:03.471766 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:03.471736    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd" event={"ID":"edcb968b-97be-45b0-8e26-39378e683676","Type":"ContainerStarted","Data":"50bcd58fc15c87b9bb68c433650fda8162f3d2e646620e8d85b527ddb89743ea"}
Oct 30 21:38:03.475501 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:03.475473    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz" event={"ID":"9d36b478-37a8-4290-bc21-dc034fb169fa","Type":"ContainerStarted","Data":"bd7b974c3f20dba487f3a1eae56ce14ab0a2a20bf44cae43afbeb581c96f7a0c"}
Oct 30 21:38:03.500734 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:03.500697    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:38:03.507139 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:03.507068    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:38:03.576208 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:03.576143    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf" podStartSLOduration=3.71139917 podStartE2EDuration="23.576122445s" totalImagesPullingTime="19.864723275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:02.799706498 +0000 UTC m=+225.471610449" watchObservedRunningTime="2026-10-30 21:38:03.576122445 +0000 UTC m=+226.248026382"
Oct 30 21:38:03.811954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:03.811839    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd" podStartSLOduration=2.030598665 podStartE2EDuration="10.811820151s" totalImagesPullingTime="8.781221486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:03.810462355 +0000 UTC m=+226.482366290" watchObservedRunningTime="2026-10-30 21:38:03.811820151 +0000 UTC m=+226.483724086"
Oct 30 21:38:04.046675 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.046618    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz" podStartSLOduration=3.183669056 podStartE2EDuration="23.046602262s" totalImagesPullingTime="19.862933206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:04.045147714 +0000 UTC m=+226.717051660" watchObservedRunningTime="2026-10-30 21:38:04.046602262 +0000 UTC m=+226.718506196"
Oct 30 21:38:04.483410 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.483373    5586 generic.go:356] "Generic (PLEG): container finished" podID="9d36b478-37a8-4290-bc21-dc034fb169fa" containerID="bd7b974c3f20dba487f3a1eae56ce14ab0a2a20bf44cae43afbeb581c96f7a0c" exitCode=0
Oct 30 21:38:04.483897 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.483639    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz" event={"ID":"9d36b478-37a8-4290-bc21-dc034fb169fa","Type":"ContainerDied","Data":"bd7b974c3f20dba487f3a1eae56ce14ab0a2a20bf44cae43afbeb581c96f7a0c"}
Oct 30 21:38:04.485198 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.485053    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:38:04.486287 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.486186    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:38:04.669041 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.668409    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"
Oct 30 21:38:04.706452 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.706431    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"
Oct 30 21:38:04.714715 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.714685    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"
Oct 30 21:38:04.716130 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.716107    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"
Oct 30 21:38:04.719507 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.719485    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9b5ccfb-59ae-463c-b93e-e60b632538e0-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9b5ccfb-59ae-463c-b93e-e60b632538e0-bundle\") pod \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\" (UID: \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\") "
Oct 30 21:38:04.719608 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.719559    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9b5ccfb-59ae-463c-b93e-e60b632538e0-kube-api-access-x4czp\" (UniqueName: \"kubernetes.io/projected/a9b5ccfb-59ae-463c-b93e-e60b632538e0-kube-api-access-x4czp\") pod \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\" (UID: \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\") "
Oct 30 21:38:04.719608 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.719587    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9b5ccfb-59ae-463c-b93e-e60b632538e0-util\" (UniqueName: \"kubernetes.io/empty-dir/a9b5ccfb-59ae-463c-b93e-e60b632538e0-util\") pod \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\" (UID: \"a9b5ccfb-59ae-463c-b93e-e60b632538e0\") "
Oct 30 21:38:04.720468 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.720438    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9b5ccfb-59ae-463c-b93e-e60b632538e0-bundle" pod "a9b5ccfb-59ae-463c-b93e-e60b632538e0" (UID: "a9b5ccfb-59ae-463c-b93e-e60b632538e0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:04.723819 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.723789    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9b5ccfb-59ae-463c-b93e-e60b632538e0-kube-api-access-x4czp" pod "a9b5ccfb-59ae-463c-b93e-e60b632538e0" (UID: "a9b5ccfb-59ae-463c-b93e-e60b632538e0"). InnerVolumeSpecName "kube-api-access-x4czp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:04.724873 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.724847    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9b5ccfb-59ae-463c-b93e-e60b632538e0-util" pod "a9b5ccfb-59ae-463c-b93e-e60b632538e0" (UID: "a9b5ccfb-59ae-463c-b93e-e60b632538e0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:04.820066 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.819985    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5522007-4832-4a97-bef2-41e82acd0553-kube-api-access-lzhbb\" (UniqueName: \"kubernetes.io/projected/a5522007-4832-4a97-bef2-41e82acd0553-kube-api-access-lzhbb\") pod \"a5522007-4832-4a97-bef2-41e82acd0553\" (UID: \"a5522007-4832-4a97-bef2-41e82acd0553\") "
Oct 30 21:38:04.820066 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.820063    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60f75b95-423a-4894-bd28-4e5262e5ca3c-bundle\" (UniqueName: \"kubernetes.io/empty-dir/60f75b95-423a-4894-bd28-4e5262e5ca3c-bundle\") pod \"60f75b95-423a-4894-bd28-4e5262e5ca3c\" (UID: \"60f75b95-423a-4894-bd28-4e5262e5ca3c\") "
Oct 30 21:38:04.820314 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.820088    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79722d5b-75c0-4443-98ea-dd16ee5a841b-kube-api-access-r4dlk\" (UniqueName: \"kubernetes.io/projected/79722d5b-75c0-4443-98ea-dd16ee5a841b-kube-api-access-r4dlk\") pod \"79722d5b-75c0-4443-98ea-dd16ee5a841b\" (UID: \"79722d5b-75c0-4443-98ea-dd16ee5a841b\") "
Oct 30 21:38:04.820314 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.820149    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60f75b95-423a-4894-bd28-4e5262e5ca3c-kube-api-access-7spm6\" (UniqueName: \"kubernetes.io/projected/60f75b95-423a-4894-bd28-4e5262e5ca3c-kube-api-access-7spm6\") pod \"60f75b95-423a-4894-bd28-4e5262e5ca3c\" (UID: \"60f75b95-423a-4894-bd28-4e5262e5ca3c\") "
Oct 30 21:38:04.820314 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.820174    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79722d5b-75c0-4443-98ea-dd16ee5a841b-util\" (UniqueName: \"kubernetes.io/empty-dir/79722d5b-75c0-4443-98ea-dd16ee5a841b-util\") pod \"79722d5b-75c0-4443-98ea-dd16ee5a841b\" (UID: \"79722d5b-75c0-4443-98ea-dd16ee5a841b\") "
Oct 30 21:38:04.820314 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.820199    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79722d5b-75c0-4443-98ea-dd16ee5a841b-bundle\" (UniqueName: \"kubernetes.io/empty-dir/79722d5b-75c0-4443-98ea-dd16ee5a841b-bundle\") pod \"79722d5b-75c0-4443-98ea-dd16ee5a841b\" (UID: \"79722d5b-75c0-4443-98ea-dd16ee5a841b\") "
Oct 30 21:38:04.820314 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.820226    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60f75b95-423a-4894-bd28-4e5262e5ca3c-util\" (UniqueName: \"kubernetes.io/empty-dir/60f75b95-423a-4894-bd28-4e5262e5ca3c-util\") pod \"60f75b95-423a-4894-bd28-4e5262e5ca3c\" (UID: \"60f75b95-423a-4894-bd28-4e5262e5ca3c\") "
Oct 30 21:38:04.820314 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.820273    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5522007-4832-4a97-bef2-41e82acd0553-util\" (UniqueName: \"kubernetes.io/empty-dir/a5522007-4832-4a97-bef2-41e82acd0553-util\") pod \"a5522007-4832-4a97-bef2-41e82acd0553\" (UID: \"a5522007-4832-4a97-bef2-41e82acd0553\") "
Oct 30 21:38:04.820314 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.820306    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5522007-4832-4a97-bef2-41e82acd0553-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5522007-4832-4a97-bef2-41e82acd0553-bundle\") pod \"a5522007-4832-4a97-bef2-41e82acd0553\" (UID: \"a5522007-4832-4a97-bef2-41e82acd0553\") "
Oct 30 21:38:04.820671 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.820581    5586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9b5ccfb-59ae-463c-b93e-e60b632538e0-bundle\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:04.820671 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.820602    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4czp\" (UniqueName: \"kubernetes.io/projected/a9b5ccfb-59ae-463c-b93e-e60b632538e0-kube-api-access-x4czp\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:04.820671 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.820616    5586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9b5ccfb-59ae-463c-b93e-e60b632538e0-util\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:04.826459 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.821216    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79722d5b-75c0-4443-98ea-dd16ee5a841b-bundle" pod "79722d5b-75c0-4443-98ea-dd16ee5a841b" (UID: "79722d5b-75c0-4443-98ea-dd16ee5a841b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:04.826459 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.821677    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5522007-4832-4a97-bef2-41e82acd0553-bundle" pod "a5522007-4832-4a97-bef2-41e82acd0553" (UID: "a5522007-4832-4a97-bef2-41e82acd0553"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:04.826459 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.822508    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60f75b95-423a-4894-bd28-4e5262e5ca3c-bundle" pod "60f75b95-423a-4894-bd28-4e5262e5ca3c" (UID: "60f75b95-423a-4894-bd28-4e5262e5ca3c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:04.826459 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.826109    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60f75b95-423a-4894-bd28-4e5262e5ca3c-util" pod "60f75b95-423a-4894-bd28-4e5262e5ca3c" (UID: "60f75b95-423a-4894-bd28-4e5262e5ca3c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:04.826791 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.826749    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79722d5b-75c0-4443-98ea-dd16ee5a841b-util" pod "79722d5b-75c0-4443-98ea-dd16ee5a841b" (UID: "79722d5b-75c0-4443-98ea-dd16ee5a841b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:04.827280 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.827255    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5522007-4832-4a97-bef2-41e82acd0553-kube-api-access-lzhbb" pod "a5522007-4832-4a97-bef2-41e82acd0553" (UID: "a5522007-4832-4a97-bef2-41e82acd0553"). InnerVolumeSpecName "kube-api-access-lzhbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:04.828335 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.828308    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60f75b95-423a-4894-bd28-4e5262e5ca3c-kube-api-access-7spm6" pod "60f75b95-423a-4894-bd28-4e5262e5ca3c" (UID: "60f75b95-423a-4894-bd28-4e5262e5ca3c"). InnerVolumeSpecName "kube-api-access-7spm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:04.828442 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.828389    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79722d5b-75c0-4443-98ea-dd16ee5a841b-kube-api-access-r4dlk" pod "79722d5b-75c0-4443-98ea-dd16ee5a841b" (UID: "79722d5b-75c0-4443-98ea-dd16ee5a841b"). InnerVolumeSpecName "kube-api-access-r4dlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:04.828491 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.828472    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5522007-4832-4a97-bef2-41e82acd0553-util" pod "a5522007-4832-4a97-bef2-41e82acd0553" (UID: "a5522007-4832-4a97-bef2-41e82acd0553"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:04.921583 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.921544    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzhbb\" (UniqueName: \"kubernetes.io/projected/a5522007-4832-4a97-bef2-41e82acd0553-kube-api-access-lzhbb\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:04.921583 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.921577    5586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60f75b95-423a-4894-bd28-4e5262e5ca3c-bundle\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:04.921583 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.921592    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4dlk\" (UniqueName: \"kubernetes.io/projected/79722d5b-75c0-4443-98ea-dd16ee5a841b-kube-api-access-r4dlk\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:04.921932 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.921605    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7spm6\" (UniqueName: \"kubernetes.io/projected/60f75b95-423a-4894-bd28-4e5262e5ca3c-kube-api-access-7spm6\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:04.921932 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.921615    5586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/79722d5b-75c0-4443-98ea-dd16ee5a841b-util\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:04.921932 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.921623    5586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/79722d5b-75c0-4443-98ea-dd16ee5a841b-bundle\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:04.921932 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.921630    5586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60f75b95-423a-4894-bd28-4e5262e5ca3c-util\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:04.921932 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.921638    5586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5522007-4832-4a97-bef2-41e82acd0553-util\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:04.921932 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:04.921645    5586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5522007-4832-4a97-bef2-41e82acd0553-bundle\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:05.493017 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.492983    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp"
Oct 30 21:38:05.493499 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.492977    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x2gjp" event={"ID":"79722d5b-75c0-4443-98ea-dd16ee5a841b","Type":"ContainerDied","Data":"ac5c03d5e7ea9bcf11f81cb3eb5dc8a2494da3c14cfe2d74f300de633401263a"}
Oct 30 21:38:05.493499 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.493133    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac5c03d5e7ea9bcf11f81cb3eb5dc8a2494da3c14cfe2d74f300de633401263a"
Oct 30 21:38:05.495165 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.495113    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x" event={"ID":"a5522007-4832-4a97-bef2-41e82acd0553","Type":"ContainerDied","Data":"311f1731c2461a84ed93e1473a357824c0235acd944e0d65725b109836e6b80f"}
Oct 30 21:38:05.495165 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.495161    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="311f1731c2461a84ed93e1473a357824c0235acd944e0d65725b109836e6b80f"
Oct 30 21:38:05.495356 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.495120    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a7fv5x"
Oct 30 21:38:05.497364 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.497344    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b"
Oct 30 21:38:05.497488 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.497422    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7192n8b" event={"ID":"60f75b95-423a-4894-bd28-4e5262e5ca3c","Type":"ContainerDied","Data":"c15f80e5edb04f80576132dc2ba792bf566da256abe5eabb9d5bed708f7f1156"}
Oct 30 21:38:05.497488 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.497466    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c15f80e5edb04f80576132dc2ba792bf566da256abe5eabb9d5bed708f7f1156"
Oct 30 21:38:05.499665 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.499626    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf" event={"ID":"a9b5ccfb-59ae-463c-b93e-e60b632538e0","Type":"ContainerDied","Data":"0b9eb3872d9fbe7ba0369094cdfe71d1282b7d7e39e19a5c3eac9704d87f3df5"}
Oct 30 21:38:05.499665 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.499663    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c68wbtf"
Oct 30 21:38:05.499811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.499672    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b9eb3872d9fbe7ba0369094cdfe71d1282b7d7e39e19a5c3eac9704d87f3df5"
Oct 30 21:38:05.502140 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.502079    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr" event={"ID":"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce","Type":"ContainerStarted","Data":"935b920621880be18ef6df274e4236a95c009923f19586ba8b0433b79717fad8"}
Oct 30 21:38:05.504255 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.504210    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4" event={"ID":"a1f9a138-94b8-490d-a1eb-32a6ece64758","Type":"ContainerStarted","Data":"f95bf9bcaad6e77c7d5afcec685ed3943cb3740473dec16f5aef425e69de8166"}
Oct 30 21:38:05.621068 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.621042    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"
Oct 30 21:38:05.728007 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.727916    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d36b478-37a8-4290-bc21-dc034fb169fa-kube-api-access-zz8nm\" (UniqueName: \"kubernetes.io/projected/9d36b478-37a8-4290-bc21-dc034fb169fa-kube-api-access-zz8nm\") pod \"9d36b478-37a8-4290-bc21-dc034fb169fa\" (UID: \"9d36b478-37a8-4290-bc21-dc034fb169fa\") "
Oct 30 21:38:05.728159 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.728010    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d36b478-37a8-4290-bc21-dc034fb169fa-util\" (UniqueName: \"kubernetes.io/empty-dir/9d36b478-37a8-4290-bc21-dc034fb169fa-util\") pod \"9d36b478-37a8-4290-bc21-dc034fb169fa\" (UID: \"9d36b478-37a8-4290-bc21-dc034fb169fa\") "
Oct 30 21:38:05.728159 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.728050    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d36b478-37a8-4290-bc21-dc034fb169fa-bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d36b478-37a8-4290-bc21-dc034fb169fa-bundle\") pod \"9d36b478-37a8-4290-bc21-dc034fb169fa\" (UID: \"9d36b478-37a8-4290-bc21-dc034fb169fa\") "
Oct 30 21:38:05.729438 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.729399    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d36b478-37a8-4290-bc21-dc034fb169fa-bundle" pod "9d36b478-37a8-4290-bc21-dc034fb169fa" (UID: "9d36b478-37a8-4290-bc21-dc034fb169fa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:05.730182 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.730151    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d36b478-37a8-4290-bc21-dc034fb169fa-kube-api-access-zz8nm" pod "9d36b478-37a8-4290-bc21-dc034fb169fa" (UID: "9d36b478-37a8-4290-bc21-dc034fb169fa"). InnerVolumeSpecName "kube-api-access-zz8nm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:05.734101 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.734076    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d36b478-37a8-4290-bc21-dc034fb169fa-util" pod "9d36b478-37a8-4290-bc21-dc034fb169fa" (UID: "9d36b478-37a8-4290-bc21-dc034fb169fa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:05.823146 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.823088    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr" podStartSLOduration=4.440338038 podStartE2EDuration="24.823073292s" totalImagesPullingTime="20.382735254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:05.822026156 +0000 UTC m=+228.493930092" watchObservedRunningTime="2026-10-30 21:38:05.823073292 +0000 UTC m=+228.494977233"
Oct 30 21:38:05.829599 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.829568    5586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d36b478-37a8-4290-bc21-dc034fb169fa-bundle\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:05.829599 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.829598    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zz8nm\" (UniqueName: \"kubernetes.io/projected/9d36b478-37a8-4290-bc21-dc034fb169fa-kube-api-access-zz8nm\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:05.829759 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.829614    5586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d36b478-37a8-4290-bc21-dc034fb169fa-util\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:05.937285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:05.937212    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4" podStartSLOduration=3.377054916 podStartE2EDuration="23.937197466s" totalImagesPullingTime="20.56014255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:05.935775171 +0000 UTC m=+228.607679108" watchObservedRunningTime="2026-10-30 21:38:05.937197466 +0000 UTC m=+228.609101400"
Oct 30 21:38:06.510677 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:06.510639    5586 generic.go:356] "Generic (PLEG): container finished" podID="431bf312-0bf1-41b4-9f30-c5cafe4bd0ce" containerID="935b920621880be18ef6df274e4236a95c009923f19586ba8b0433b79717fad8" exitCode=0
Oct 30 21:38:06.511986 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:06.510720    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr" event={"ID":"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce","Type":"ContainerDied","Data":"935b920621880be18ef6df274e4236a95c009923f19586ba8b0433b79717fad8"}
Oct 30 21:38:06.512672 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:06.512654    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz"
Oct 30 21:38:06.512767 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:06.512676    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckwcxz" event={"ID":"9d36b478-37a8-4290-bc21-dc034fb169fa","Type":"ContainerDied","Data":"e7042b1891dfce0f2205003359722cccdc16c9a8f856b8f2eac183a4d5c01092"}
Oct 30 21:38:06.512767 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:06.512712    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7042b1891dfce0f2205003359722cccdc16c9a8f856b8f2eac183a4d5c01092"
Oct 30 21:38:07.639375 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:07.639350    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"
Oct 30 21:38:07.744468 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:07.744431    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-bundle\" (UniqueName: \"kubernetes.io/empty-dir/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-bundle\") pod \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\" (UID: \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\") "
Oct 30 21:38:07.744645 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:07.744545    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-kube-api-access-lfwk6\" (UniqueName: \"kubernetes.io/projected/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-kube-api-access-lfwk6\") pod \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\" (UID: \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\") "
Oct 30 21:38:07.744645 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:07.744573    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-util\" (UniqueName: \"kubernetes.io/empty-dir/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-util\") pod \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\" (UID: \"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce\") "
Oct 30 21:38:07.745162 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:07.745127    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-bundle" pod "431bf312-0bf1-41b4-9f30-c5cafe4bd0ce" (UID: "431bf312-0bf1-41b4-9f30-c5cafe4bd0ce"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:07.746661 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:07.746632    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-kube-api-access-lfwk6" pod "431bf312-0bf1-41b4-9f30-c5cafe4bd0ce" (UID: "431bf312-0bf1-41b4-9f30-c5cafe4bd0ce"). InnerVolumeSpecName "kube-api-access-lfwk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:07.748867 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:07.748841    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-util" pod "431bf312-0bf1-41b4-9f30-c5cafe4bd0ce" (UID: "431bf312-0bf1-41b4-9f30-c5cafe4bd0ce"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:07.845953 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:07.845851    5586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-bundle\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:07.845953 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:07.845919    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfwk6\" (UniqueName: \"kubernetes.io/projected/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-kube-api-access-lfwk6\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:07.845953 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:07.845934    5586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/431bf312-0bf1-41b4-9f30-c5cafe4bd0ce-util\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:08.522245 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:08.522208    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr" event={"ID":"431bf312-0bf1-41b4-9f30-c5cafe4bd0ce","Type":"ContainerDied","Data":"d6e4db0c0159ccd5c4fbfda55e53d38cbda2b87b120e2f521b88208673ff3ca8"}
Oct 30 21:38:08.522427 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:08.522280    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6e4db0c0159ccd5c4fbfda55e53d38cbda2b87b120e2f521b88208673ff3ca8"
Oct 30 21:38:08.522427 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:08.522284    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c4tqwr"
Oct 30 21:38:08.659762 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:08.659731    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:09.663267 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:09.663210    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:10.662773 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:10.662740    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:14.446604 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446565    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"]
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446877    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a9b5ccfb-59ae-463c-b93e-e60b632538e0" containerName="util"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446889    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9b5ccfb-59ae-463c-b93e-e60b632538e0" containerName="util"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446903    5586 cpu_manager.go:414] "containerMap: removing container" podUID="9d36b478-37a8-4290-bc21-dc034fb169fa" containerName="util"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446910    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d36b478-37a8-4290-bc21-dc034fb169fa" containerName="util"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446917    5586 cpu_manager.go:414] "containerMap: removing container" podUID="79722d5b-75c0-4443-98ea-dd16ee5a841b" containerName="util"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446923    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79722d5b-75c0-4443-98ea-dd16ee5a841b" containerName="util"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446931    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a5522007-4832-4a97-bef2-41e82acd0553" containerName="util"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446935    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5522007-4832-4a97-bef2-41e82acd0553" containerName="util"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446941    5586 cpu_manager.go:414] "containerMap: removing container" podUID="9d36b478-37a8-4290-bc21-dc034fb169fa" containerName="extract"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446945    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d36b478-37a8-4290-bc21-dc034fb169fa" containerName="extract"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446952    5586 cpu_manager.go:414] "containerMap: removing container" podUID="60f75b95-423a-4894-bd28-4e5262e5ca3c" containerName="pull"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446956    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60f75b95-423a-4894-bd28-4e5262e5ca3c" containerName="pull"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446963    5586 cpu_manager.go:414] "containerMap: removing container" podUID="431bf312-0bf1-41b4-9f30-c5cafe4bd0ce" containerName="util"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446967    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="431bf312-0bf1-41b4-9f30-c5cafe4bd0ce" containerName="util"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446973    5586 cpu_manager.go:414] "containerMap: removing container" podUID="60f75b95-423a-4894-bd28-4e5262e5ca3c" containerName="extract"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446977    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60f75b95-423a-4894-bd28-4e5262e5ca3c" containerName="extract"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446985    5586 cpu_manager.go:414] "containerMap: removing container" podUID="79722d5b-75c0-4443-98ea-dd16ee5a841b" containerName="extract"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446991    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79722d5b-75c0-4443-98ea-dd16ee5a841b" containerName="extract"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.446997    5586 cpu_manager.go:414] "containerMap: removing container" podUID="60f75b95-423a-4894-bd28-4e5262e5ca3c" containerName="util"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447001    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60f75b95-423a-4894-bd28-4e5262e5ca3c" containerName="util"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447007    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a9b5ccfb-59ae-463c-b93e-e60b632538e0" containerName="extract"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447012    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9b5ccfb-59ae-463c-b93e-e60b632538e0" containerName="extract"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447023    5586 cpu_manager.go:414] "containerMap: removing container" podUID="431bf312-0bf1-41b4-9f30-c5cafe4bd0ce" containerName="pull"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447027    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="431bf312-0bf1-41b4-9f30-c5cafe4bd0ce" containerName="pull"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447036    5586 cpu_manager.go:414] "containerMap: removing container" podUID="431bf312-0bf1-41b4-9f30-c5cafe4bd0ce" containerName="extract"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447040    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="431bf312-0bf1-41b4-9f30-c5cafe4bd0ce" containerName="extract"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447046    5586 cpu_manager.go:414] "containerMap: removing container" podUID="79722d5b-75c0-4443-98ea-dd16ee5a841b" containerName="pull"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447050    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79722d5b-75c0-4443-98ea-dd16ee5a841b" containerName="pull"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447056    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a5522007-4832-4a97-bef2-41e82acd0553" containerName="pull"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447060    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5522007-4832-4a97-bef2-41e82acd0553" containerName="pull"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447065    5586 cpu_manager.go:414] "containerMap: removing container" podUID="9d36b478-37a8-4290-bc21-dc034fb169fa" containerName="pull"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447069    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d36b478-37a8-4290-bc21-dc034fb169fa" containerName="pull"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447078    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a9b5ccfb-59ae-463c-b93e-e60b632538e0" containerName="pull"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447083    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9b5ccfb-59ae-463c-b93e-e60b632538e0" containerName="pull"
Oct 30 21:38:14.447069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447089    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a5522007-4832-4a97-bef2-41e82acd0553" containerName="extract"
Oct 30 21:38:14.448087 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447094    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5522007-4832-4a97-bef2-41e82acd0553" containerName="extract"
Oct 30 21:38:14.448087 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447153    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d36b478-37a8-4290-bc21-dc034fb169fa" containerName="extract"
Oct 30 21:38:14.448087 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447161    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="79722d5b-75c0-4443-98ea-dd16ee5a841b" containerName="extract"
Oct 30 21:38:14.448087 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447167    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="60f75b95-423a-4894-bd28-4e5262e5ca3c" containerName="extract"
Oct 30 21:38:14.448087 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447172    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9b5ccfb-59ae-463c-b93e-e60b632538e0" containerName="extract"
Oct 30 21:38:14.448087 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447179    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="431bf312-0bf1-41b4-9f30-c5cafe4bd0ce" containerName="extract"
Oct 30 21:38:14.448087 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.447185    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5522007-4832-4a97-bef2-41e82acd0553" containerName="extract"
Oct 30 21:38:14.448969 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.448954    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"
Oct 30 21:38:14.504155 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.504118    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9689\" (UniqueName: \"kubernetes.io/projected/b77cc961-5c28-4437-83fe-037e93fecdb6-kube-api-access-p9689\") pod \"kuadrant-operator-controller-manager-cf8469d7b-6h2cv\" (UID: \"b77cc961-5c28-4437-83fe-037e93fecdb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"
Oct 30 21:38:14.504338 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.504163    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b77cc961-5c28-4437-83fe-037e93fecdb6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-6h2cv\" (UID: \"b77cc961-5c28-4437-83fe-037e93fecdb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"
Oct 30 21:38:14.551346 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.551311    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 21:38:14.551544 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.551363    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 21:38:14.551544 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.551368    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lsrdr\""
Oct 30 21:38:14.552345 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.552328    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"]
Oct 30 21:38:14.604766 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.604728    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9689\" (UniqueName: \"kubernetes.io/projected/b77cc961-5c28-4437-83fe-037e93fecdb6-kube-api-access-p9689\") pod \"kuadrant-operator-controller-manager-cf8469d7b-6h2cv\" (UID: \"b77cc961-5c28-4437-83fe-037e93fecdb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"
Oct 30 21:38:14.604894 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.604775    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b77cc961-5c28-4437-83fe-037e93fecdb6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-6h2cv\" (UID: \"b77cc961-5c28-4437-83fe-037e93fecdb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"
Oct 30 21:38:14.605119 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.605103    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b77cc961-5c28-4437-83fe-037e93fecdb6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-6h2cv\" (UID: \"b77cc961-5c28-4437-83fe-037e93fecdb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"
Oct 30 21:38:14.665337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.665308    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9689\" (UniqueName: \"kubernetes.io/projected/b77cc961-5c28-4437-83fe-037e93fecdb6-kube-api-access-p9689\") pod \"kuadrant-operator-controller-manager-cf8469d7b-6h2cv\" (UID: \"b77cc961-5c28-4437-83fe-037e93fecdb6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"
Oct 30 21:38:14.760684 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:14.760588    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"
Oct 30 21:38:15.243432 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:38:15.243399    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb77cc961_5c28_4437_83fe_037e93fecdb6.slice/crio-a2c20225593a1104c4babe85369395ac7e31ad7e92bf970d354b52d26c13fdea WatchSource:0}: Error finding container a2c20225593a1104c4babe85369395ac7e31ad7e92bf970d354b52d26c13fdea: Status 404 returned error can't find the container with id a2c20225593a1104c4babe85369395ac7e31ad7e92bf970d354b52d26c13fdea
Oct 30 21:38:15.243567 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:15.243439    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"]
Oct 30 21:38:15.551073 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:15.551001    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 21:38:15.551655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:15.551632    5586 generic.go:356] "Generic (PLEG): container finished" podID="a1f9a138-94b8-490d-a1eb-32a6ece64758" containerID="f95bf9bcaad6e77c7d5afcec685ed3943cb3740473dec16f5aef425e69de8166" exitCode=1
Oct 30 21:38:15.551724 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:15.551704    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4" event={"ID":"a1f9a138-94b8-490d-a1eb-32a6ece64758","Type":"ContainerDied","Data":"f95bf9bcaad6e77c7d5afcec685ed3943cb3740473dec16f5aef425e69de8166"}
Oct 30 21:38:15.552972 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:15.552947    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv" event={"ID":"b77cc961-5c28-4437-83fe-037e93fecdb6","Type":"ContainerStarted","Data":"a2c20225593a1104c4babe85369395ac7e31ad7e92bf970d354b52d26c13fdea"}
Oct 30 21:38:15.667678 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:15.667640    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:15.906378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:15.906342    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:16.085068 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:16.085027    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:16.507772 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:16.507737    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:16.673722 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:16.673701    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 21:38:16.675985 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:16.675966    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"
Oct 30 21:38:16.823414 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:16.823337    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1f9a138-94b8-490d-a1eb-32a6ece64758-kube-api-access-2wp9z\" (UniqueName: \"kubernetes.io/projected/a1f9a138-94b8-490d-a1eb-32a6ece64758-kube-api-access-2wp9z\") pod \"a1f9a138-94b8-490d-a1eb-32a6ece64758\" (UID: \"a1f9a138-94b8-490d-a1eb-32a6ece64758\") "
Oct 30 21:38:16.823575 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:16.823458    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1f9a138-94b8-490d-a1eb-32a6ece64758-util\" (UniqueName: \"kubernetes.io/empty-dir/a1f9a138-94b8-490d-a1eb-32a6ece64758-util\") pod \"a1f9a138-94b8-490d-a1eb-32a6ece64758\" (UID: \"a1f9a138-94b8-490d-a1eb-32a6ece64758\") "
Oct 30 21:38:16.823575 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:16.823492    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1f9a138-94b8-490d-a1eb-32a6ece64758-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1f9a138-94b8-490d-a1eb-32a6ece64758-bundle\") pod \"a1f9a138-94b8-490d-a1eb-32a6ece64758\" (UID: \"a1f9a138-94b8-490d-a1eb-32a6ece64758\") "
Oct 30 21:38:16.823918 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:16.823887    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1f9a138-94b8-490d-a1eb-32a6ece64758-bundle" pod "a1f9a138-94b8-490d-a1eb-32a6ece64758" (UID: "a1f9a138-94b8-490d-a1eb-32a6ece64758"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:16.825334 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:16.825309    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1f9a138-94b8-490d-a1eb-32a6ece64758-kube-api-access-2wp9z" pod "a1f9a138-94b8-490d-a1eb-32a6ece64758" (UID: "a1f9a138-94b8-490d-a1eb-32a6ece64758"). InnerVolumeSpecName "kube-api-access-2wp9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:16.828695 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:16.828672    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1f9a138-94b8-490d-a1eb-32a6ece64758-util" pod "a1f9a138-94b8-490d-a1eb-32a6ece64758" (UID: "a1f9a138-94b8-490d-a1eb-32a6ece64758"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:16.924549 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:16.924511    5586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1f9a138-94b8-490d-a1eb-32a6ece64758-util\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:16.924549 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:16.924542    5586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1f9a138-94b8-490d-a1eb-32a6ece64758-bundle\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:16.924549 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:16.924553    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wp9z\" (UniqueName: \"kubernetes.io/projected/a1f9a138-94b8-490d-a1eb-32a6ece64758-kube-api-access-2wp9z\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:17.282404 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.282306    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:17.532772 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.532626    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:17.564534 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.564484    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 21:38:17.565369 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.565341    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4" event={"ID":"a1f9a138-94b8-490d-a1eb-32a6ece64758","Type":"ContainerDied","Data":"73b49fda24c54b1930aa4319e96c38b75e1d2c34e74f55981dbd97e3f6a78e63"}
Oct 30 21:38:17.565522 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.565397    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73b49fda24c54b1930aa4319e96c38b75e1d2c34e74f55981dbd97e3f6a78e63"
Oct 30 21:38:17.565522 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.565483    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4"
Oct 30 21:38:17.717609 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.716041    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"]
Oct 30 21:38:17.717609 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.716449    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a1f9a138-94b8-490d-a1eb-32a6ece64758" containerName="pull"
Oct 30 21:38:17.717609 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.716465    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1f9a138-94b8-490d-a1eb-32a6ece64758" containerName="pull"
Oct 30 21:38:17.717609 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.716478    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a1f9a138-94b8-490d-a1eb-32a6ece64758" containerName="util"
Oct 30 21:38:17.717609 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.716483    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1f9a138-94b8-490d-a1eb-32a6ece64758" containerName="util"
Oct 30 21:38:17.717609 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.716493    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a1f9a138-94b8-490d-a1eb-32a6ece64758" containerName="extract"
Oct 30 21:38:17.717609 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.716499    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1f9a138-94b8-490d-a1eb-32a6ece64758" containerName="extract"
Oct 30 21:38:17.717609 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.716550    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1f9a138-94b8-490d-a1eb-32a6ece64758" containerName="extract"
Oct 30 21:38:17.718696 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.718667    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.769204 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.769104    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"]
Oct 30 21:38:17.817091 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.816992    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 21:38:17.817806 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.817780    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 21:38:17.817977 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.817878    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 21:38:17.817977 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.817927    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 21:38:17.818097 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.818021    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-xkqbq\""
Oct 30 21:38:17.818097 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.818073    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 21:38:17.834665 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.834624    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ss5t\" (UniqueName: \"kubernetes.io/projected/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-kube-api-access-2ss5t\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.834832 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.834755    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.834888 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.834830    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-metrics-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.835013 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.834982    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-tmp\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.835146 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.835055    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-manifests\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.835146 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.835124    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.885551 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.885510    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:17.936622 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.936582    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-tmp\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.936791 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.936645    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-manifests\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.936791 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.936683    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.936791 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.936713    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ss5t\" (UniqueName: \"kubernetes.io/projected/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-kube-api-access-2ss5t\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.936791 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.936760    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.937015 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.936806    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-metrics-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.937945 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.937908    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-manifests\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.940350 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.939813    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-tmp\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.940350 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.940309    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.940518 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.940348    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-metrics-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:17.940518 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:17.940473    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:18.004929 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.004888    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ss5t\" (UniqueName: \"kubernetes.io/projected/b8b9e41f-41fa-40d9-b1a8-d6e12596eda2-kube-api-access-2ss5t\") pod \"opendatahub-operator-controller-manager-7c746f769b-f5lgh\" (UID: \"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:18.035637 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.035596    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:18.065576 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.065534    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:18.233471 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.233408    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:18.368136 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:38:18.368038    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:38:18.387261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.385966    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-xx2qj"]
Oct 30 21:38:18.390245 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.390211    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xx2qj"
Oct 30 21:38:18.435648 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.435608    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:18.443478 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.443447    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 30 21:38:18.447958 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.447913    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7t49s\""
Oct 30 21:38:18.457858 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.457820    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-xx2qj"]
Oct 30 21:38:18.542132 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.542095    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prszp\" (UniqueName: \"kubernetes.io/projected/692e7194-b0df-4f38-8ea9-4fd9abff8e97-kube-api-access-prszp\") pod \"dns-operator-controller-manager-8557cbdb84-xx2qj\" (UID: \"692e7194-b0df-4f38-8ea9-4fd9abff8e97\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xx2qj"
Oct 30 21:38:18.555407 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.555370    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"]
Oct 30 21:38:18.579523 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.579489    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh" event={"ID":"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2","Type":"ContainerStarted","Data":"075a2910c55bdaf3813291b5528b710168941214a56c05b92dc2c92a9909f966"}
Oct 30 21:38:18.643332 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.643244    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prszp\" (UniqueName: \"kubernetes.io/projected/692e7194-b0df-4f38-8ea9-4fd9abff8e97-kube-api-access-prszp\") pod \"dns-operator-controller-manager-8557cbdb84-xx2qj\" (UID: \"692e7194-b0df-4f38-8ea9-4fd9abff8e97\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xx2qj"
Oct 30 21:38:18.706255 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.706145    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prszp\" (UniqueName: \"kubernetes.io/projected/692e7194-b0df-4f38-8ea9-4fd9abff8e97-kube-api-access-prszp\") pod \"dns-operator-controller-manager-8557cbdb84-xx2qj\" (UID: \"692e7194-b0df-4f38-8ea9-4fd9abff8e97\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xx2qj"
Oct 30 21:38:18.865767 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:18.865715    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:19.003229 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:19.003185    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xx2qj"
Oct 30 21:38:19.394898 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:19.394811    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-xx2qj"]
Oct 30 21:38:19.399420 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:38:19.399384    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod692e7194_b0df_4f38_8ea9_4fd9abff8e97.slice/crio-b72ef73119210f07fa559ee05bb078d62195da7727af6d7b0cd67a724c868186 WatchSource:0}: Error finding container b72ef73119210f07fa559ee05bb078d62195da7727af6d7b0cd67a724c868186: Status 404 returned error can't find the container with id b72ef73119210f07fa559ee05bb078d62195da7727af6d7b0cd67a724c868186
Oct 30 21:38:19.503438 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:19.503397    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:19.595038 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:19.594956    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xx2qj" event={"ID":"692e7194-b0df-4f38-8ea9-4fd9abff8e97","Type":"ContainerStarted","Data":"b72ef73119210f07fa559ee05bb078d62195da7727af6d7b0cd67a724c868186"}
Oct 30 21:38:20.651094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:20.651057    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:20.832423 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:20.832372    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:20.992057 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:20.992020    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:21.609565 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:21.609522    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv" event={"ID":"b77cc961-5c28-4437-83fe-037e93fecdb6","Type":"ContainerStarted","Data":"d0eb92da15768d498391ebecf5e4f943ec2b941358a11c6113730d7bfcc65fef"}
Oct 30 21:38:21.610447 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:21.610421    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"
Oct 30 21:38:21.833196 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:21.833140    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv" podStartSLOduration=2.3861144530000002 podStartE2EDuration="7.833120674s" totalImagesPullingTime="5.447006221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:21.782512813 +0000 UTC m=+244.454416748" watchObservedRunningTime="2026-10-30 21:38:21.833120674 +0000 UTC m=+244.505024616"
Oct 30 21:38:22.616206 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:22.616167    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh" event={"ID":"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2","Type":"ContainerStarted","Data":"fdbdf645e336f4063c5184cec368ff30225cee5a2af0ad8f67509505f883f23a"}
Oct 30 21:38:22.617824 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:22.617796    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xx2qj" event={"ID":"692e7194-b0df-4f38-8ea9-4fd9abff8e97","Type":"ContainerStarted","Data":"d8148a7f93f9782645182455586d7d3b06bd6951636406e0e4709237c8a54ba4"}
Oct 30 21:38:22.931653 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:22.931589    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xx2qj" podStartSLOduration=2.19416353 podStartE2EDuration="4.931568017s" totalImagesPullingTime="2.737404487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:22.931515325 +0000 UTC m=+245.603419261" watchObservedRunningTime="2026-10-30 21:38:22.931568017 +0000 UTC m=+245.603471955"
Oct 30 21:38:23.623755 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:23.623720    5586 generic.go:356] "Generic (PLEG): container finished" podID="b8b9e41f-41fa-40d9-b1a8-d6e12596eda2" containerID="fdbdf645e336f4063c5184cec368ff30225cee5a2af0ad8f67509505f883f23a" exitCode=0
Oct 30 21:38:23.623929 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:23.623822    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh" event={"ID":"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2","Type":"ContainerDied","Data":"fdbdf645e336f4063c5184cec368ff30225cee5a2af0ad8f67509505f883f23a"}
Oct 30 21:38:23.624072 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:23.624054    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xx2qj"
Oct 30 21:38:23.714823 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:23.714784    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:23.917818 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:23.917781    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:24.322936 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:24.322853    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:24.630916 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:24.630819    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh" event={"ID":"b8b9e41f-41fa-40d9-b1a8-d6e12596eda2","Type":"ContainerStarted","Data":"ac036ccc9f4d188ffcb23a3a504172cd8037036baeece5018b8920e36132d0b4"}
Oct 30 21:38:24.768868 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:24.768809    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh" podStartSLOduration=2.858510184 podStartE2EDuration="7.768793528s" totalImagesPullingTime="4.38728205s" totalInitContainerRuntime="523.001294ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:24.767985552 +0000 UTC m=+247.439889480" watchObservedRunningTime="2026-10-30 21:38:24.768793528 +0000 UTC m=+247.440697462"
Oct 30 21:38:25.055993 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.055956    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"]
Oct 30 21:38:25.058260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.058226    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"
Oct 30 21:38:25.065657 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.065617    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:25.115107 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.115072    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 21:38:25.115107 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.115079    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-9fs5n\""
Oct 30 21:38:25.116041 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.116021    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 21:38:25.132640 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.132609    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"]
Oct 30 21:38:25.206668 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.206637    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d4af674-8580-417f-b34d-6d593c9f05e3-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c\" (UID: \"4d4af674-8580-417f-b34d-6d593c9f05e3\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"
Oct 30 21:38:25.206848 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.206759    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6kg8\" (UniqueName: \"kubernetes.io/projected/4d4af674-8580-417f-b34d-6d593c9f05e3-kube-api-access-g6kg8\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c\" (UID: \"4d4af674-8580-417f-b34d-6d593c9f05e3\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"
Oct 30 21:38:25.206848 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.206798    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d4af674-8580-417f-b34d-6d593c9f05e3-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c\" (UID: \"4d4af674-8580-417f-b34d-6d593c9f05e3\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"
Oct 30 21:38:25.307932 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.307840    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6kg8\" (UniqueName: \"kubernetes.io/projected/4d4af674-8580-417f-b34d-6d593c9f05e3-kube-api-access-g6kg8\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c\" (UID: \"4d4af674-8580-417f-b34d-6d593c9f05e3\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"
Oct 30 21:38:25.307932 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.307889    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d4af674-8580-417f-b34d-6d593c9f05e3-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c\" (UID: \"4d4af674-8580-417f-b34d-6d593c9f05e3\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"
Oct 30 21:38:25.307932 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.307915    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d4af674-8580-417f-b34d-6d593c9f05e3-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c\" (UID: \"4d4af674-8580-417f-b34d-6d593c9f05e3\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"
Oct 30 21:38:25.308293 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.308263    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d4af674-8580-417f-b34d-6d593c9f05e3-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c\" (UID: \"4d4af674-8580-417f-b34d-6d593c9f05e3\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"
Oct 30 21:38:25.308429 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.308337    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d4af674-8580-417f-b34d-6d593c9f05e3-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c\" (UID: \"4d4af674-8580-417f-b34d-6d593c9f05e3\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"
Oct 30 21:38:25.366932 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.366889    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6kg8\" (UniqueName: \"kubernetes.io/projected/4d4af674-8580-417f-b34d-6d593c9f05e3-kube-api-access-g6kg8\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c\" (UID: \"4d4af674-8580-417f-b34d-6d593c9f05e3\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"
Oct 30 21:38:25.371068 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.371043    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"
Oct 30 21:38:25.635741 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.635663    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:25.749385 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:25.749352    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"]
Oct 30 21:38:25.752614 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:38:25.752577    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d4af674_8580_417f_b34d_6d593c9f05e3.slice/crio-108fea0a56b04b31cf077db1bda96b4ec7729f5355a72725294ea8276c20c1ae WatchSource:0}: Error finding container 108fea0a56b04b31cf077db1bda96b4ec7729f5355a72725294ea8276c20c1ae: Status 404 returned error can't find the container with id 108fea0a56b04b31cf077db1bda96b4ec7729f5355a72725294ea8276c20c1ae
Oct 30 21:38:26.386873 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.386831    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:26.403764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.403735    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq"]
Oct 30 21:38:26.406035 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.406014    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq"
Oct 30 21:38:26.474334 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.474302    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Oct 30 21:38:26.479717 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.479674    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hfdpc\""
Oct 30 21:38:26.479956 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.479751    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Oct 30 21:38:26.481177 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.481150    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq"]
Oct 30 21:38:26.518988 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.518950    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/54e53152-2818-40cb-a054-c313ce649d24-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-q66hq\" (UID: \"54e53152-2818-40cb-a054-c313ce649d24\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq"
Oct 30 21:38:26.519209 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.519020    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnxfp\" (UniqueName: \"kubernetes.io/projected/54e53152-2818-40cb-a054-c313ce649d24-kube-api-access-bnxfp\") pod \"kuadrant-console-plugin-66c4fb575d-q66hq\" (UID: \"54e53152-2818-40cb-a054-c313ce649d24\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq"
Oct 30 21:38:26.519209 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.519096    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/54e53152-2818-40cb-a054-c313ce649d24-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-q66hq\" (UID: \"54e53152-2818-40cb-a054-c313ce649d24\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq"
Oct 30 21:38:26.607446 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.607407    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:26.620279 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.620249    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/54e53152-2818-40cb-a054-c313ce649d24-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-q66hq\" (UID: \"54e53152-2818-40cb-a054-c313ce649d24\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq"
Oct 30 21:38:26.620452 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.620296    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnxfp\" (UniqueName: \"kubernetes.io/projected/54e53152-2818-40cb-a054-c313ce649d24-kube-api-access-bnxfp\") pod \"kuadrant-console-plugin-66c4fb575d-q66hq\" (UID: \"54e53152-2818-40cb-a054-c313ce649d24\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq"
Oct 30 21:38:26.620452 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.620335    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/54e53152-2818-40cb-a054-c313ce649d24-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-q66hq\" (UID: \"54e53152-2818-40cb-a054-c313ce649d24\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq"
Oct 30 21:38:26.620947 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.620926    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/54e53152-2818-40cb-a054-c313ce649d24-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-q66hq\" (UID: \"54e53152-2818-40cb-a054-c313ce649d24\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq"
Oct 30 21:38:26.622804 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.622772    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/54e53152-2818-40cb-a054-c313ce649d24-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-q66hq\" (UID: \"54e53152-2818-40cb-a054-c313ce649d24\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq"
Oct 30 21:38:26.641910 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.641827    5586 generic.go:356] "Generic (PLEG): container finished" podID="4d4af674-8580-417f-b34d-6d593c9f05e3" containerID="1f2505f6952791ffb178a61b40df15bfbfb60cb1b148d51318a6715211bdeb52" exitCode=0
Oct 30 21:38:26.642069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.641912    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c" event={"ID":"4d4af674-8580-417f-b34d-6d593c9f05e3","Type":"ContainerDied","Data":"1f2505f6952791ffb178a61b40df15bfbfb60cb1b148d51318a6715211bdeb52"}
Oct 30 21:38:26.642069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.641975    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c" event={"ID":"4d4af674-8580-417f-b34d-6d593c9f05e3","Type":"ContainerStarted","Data":"108fea0a56b04b31cf077db1bda96b4ec7729f5355a72725294ea8276c20c1ae"}
Oct 30 21:38:26.680748 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.680722    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnxfp\" (UniqueName: \"kubernetes.io/projected/54e53152-2818-40cb-a054-c313ce649d24-kube-api-access-bnxfp\") pod \"kuadrant-console-plugin-66c4fb575d-q66hq\" (UID: \"54e53152-2818-40cb-a054-c313ce649d24\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq"
Oct 30 21:38:26.727650 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.727617    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq"
Oct 30 21:38:26.828916 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:26.828871    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:27.114757 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:27.114717    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq"]
Oct 30 21:38:27.118748 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:38:27.118716    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54e53152_2818_40cb_a054_c313ce649d24.slice/crio-cc3bea18c2f87d8dab731dfcfa20a4c0b48d227dcd194b0ea09ba3c914241f86 WatchSource:0}: Error finding container cc3bea18c2f87d8dab731dfcfa20a4c0b48d227dcd194b0ea09ba3c914241f86: Status 404 returned error can't find the container with id cc3bea18c2f87d8dab731dfcfa20a4c0b48d227dcd194b0ea09ba3c914241f86
Oct 30 21:38:27.647795 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:27.647745    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq" event={"ID":"54e53152-2818-40cb-a054-c313ce649d24","Type":"ContainerStarted","Data":"cc3bea18c2f87d8dab731dfcfa20a4c0b48d227dcd194b0ea09ba3c914241f86"}
Oct 30 21:38:27.650163 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:27.650130    5586 generic.go:356] "Generic (PLEG): container finished" podID="4d4af674-8580-417f-b34d-6d593c9f05e3" containerID="0d02ddd56f1946a6534163222de01b11d0b5f58c9de850c8028ed1f67c609ae1" exitCode=0
Oct 30 21:38:27.650299 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:27.650206    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c" event={"ID":"4d4af674-8580-417f-b34d-6d593c9f05e3","Type":"ContainerDied","Data":"0d02ddd56f1946a6534163222de01b11d0b5f58c9de850c8028ed1f67c609ae1"}
Oct 30 21:38:28.657092 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:28.657058    5586 generic.go:356] "Generic (PLEG): container finished" podID="4d4af674-8580-417f-b34d-6d593c9f05e3" containerID="9ee4f879997d8a94eb8f52d33c1dbf0953cef9e5ad41478406eca7310fb8aee6" exitCode=0
Oct 30 21:38:28.657590 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:28.657117    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c" event={"ID":"4d4af674-8580-417f-b34d-6d593c9f05e3","Type":"ContainerDied","Data":"9ee4f879997d8a94eb8f52d33c1dbf0953cef9e5ad41478406eca7310fb8aee6"}
Oct 30 21:38:28.731507 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:28.731464    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:28.908794 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:28.908694    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:29.100563 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.100524    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:29.358149 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.358058    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:29.804682 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.804648    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"
Oct 30 21:38:29.949309 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.948331    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d4af674-8580-417f-b34d-6d593c9f05e3-bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d4af674-8580-417f-b34d-6d593c9f05e3-bundle\") pod \"4d4af674-8580-417f-b34d-6d593c9f05e3\" (UID: \"4d4af674-8580-417f-b34d-6d593c9f05e3\") "
Oct 30 21:38:29.949309 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.948422    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d4af674-8580-417f-b34d-6d593c9f05e3-util\" (UniqueName: \"kubernetes.io/empty-dir/4d4af674-8580-417f-b34d-6d593c9f05e3-util\") pod \"4d4af674-8580-417f-b34d-6d593c9f05e3\" (UID: \"4d4af674-8580-417f-b34d-6d593c9f05e3\") "
Oct 30 21:38:29.949309 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.948468    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d4af674-8580-417f-b34d-6d593c9f05e3-kube-api-access-g6kg8\" (UniqueName: \"kubernetes.io/projected/4d4af674-8580-417f-b34d-6d593c9f05e3-kube-api-access-g6kg8\") pod \"4d4af674-8580-417f-b34d-6d593c9f05e3\" (UID: \"4d4af674-8580-417f-b34d-6d593c9f05e3\") "
Oct 30 21:38:29.950068 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.950017    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d4af674-8580-417f-b34d-6d593c9f05e3-bundle" pod "4d4af674-8580-417f-b34d-6d593c9f05e3" (UID: "4d4af674-8580-417f-b34d-6d593c9f05e3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.957596 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.957557    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d4af674-8580-417f-b34d-6d593c9f05e3-kube-api-access-g6kg8" pod "4d4af674-8580-417f-b34d-6d593c9f05e3" (UID: "4d4af674-8580-417f-b34d-6d593c9f05e3"). InnerVolumeSpecName "kube-api-access-g6kg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:29.958031 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.957991    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d4af674-8580-417f-b34d-6d593c9f05e3-util" pod "4d4af674-8580-417f-b34d-6d593c9f05e3" (UID: "4d4af674-8580-417f-b34d-6d593c9f05e3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:29.988428 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.986706    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd"]
Oct 30 21:38:29.988428 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.987165    5586 cpu_manager.go:414] "containerMap: removing container" podUID="4d4af674-8580-417f-b34d-6d593c9f05e3" containerName="pull"
Oct 30 21:38:29.988428 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.987181    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d4af674-8580-417f-b34d-6d593c9f05e3" containerName="pull"
Oct 30 21:38:29.988428 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.987213    5586 cpu_manager.go:414] "containerMap: removing container" podUID="4d4af674-8580-417f-b34d-6d593c9f05e3" containerName="util"
Oct 30 21:38:29.988428 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.987220    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d4af674-8580-417f-b34d-6d593c9f05e3" containerName="util"
Oct 30 21:38:29.988428 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.987257    5586 cpu_manager.go:414] "containerMap: removing container" podUID="4d4af674-8580-417f-b34d-6d593c9f05e3" containerName="extract"
Oct 30 21:38:29.988428 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.987265    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d4af674-8580-417f-b34d-6d593c9f05e3" containerName="extract"
Oct 30 21:38:29.988428 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:29.987361    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d4af674-8580-417f-b34d-6d593c9f05e3" containerName="extract"
Oct 30 21:38:30.015827 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.015792    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd"
Oct 30 21:38:30.051990 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.051909    5586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d4af674-8580-417f-b34d-6d593c9f05e3-bundle\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:30.052162 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.051995    5586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d4af674-8580-417f-b34d-6d593c9f05e3-util\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:30.052162 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.052058    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6kg8\" (UniqueName: \"kubernetes.io/projected/4d4af674-8580-417f-b34d-6d593c9f05e3-kube-api-access-g6kg8\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:30.069729 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.069691    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 21:38:30.075879 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.075831    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:38:30.076649 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.076625    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-djhwh\""
Oct 30 21:38:30.120858 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.120673    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd"]
Oct 30 21:38:30.152936 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.152896    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cgs5\" (UniqueName: \"kubernetes.io/projected/3f72be6f-6647-49dc-a7dc-2d722f870fed-kube-api-access-6cgs5\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd\" (UID: \"3f72be6f-6647-49dc-a7dc-2d722f870fed\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd"
Oct 30 21:38:30.153102 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.152945    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f72be6f-6647-49dc-a7dc-2d722f870fed-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd\" (UID: \"3f72be6f-6647-49dc-a7dc-2d722f870fed\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd"
Oct 30 21:38:30.254079 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.253961    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f72be6f-6647-49dc-a7dc-2d722f870fed-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd\" (UID: \"3f72be6f-6647-49dc-a7dc-2d722f870fed\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd"
Oct 30 21:38:30.254225 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.254108    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cgs5\" (UniqueName: \"kubernetes.io/projected/3f72be6f-6647-49dc-a7dc-2d722f870fed-kube-api-access-6cgs5\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd\" (UID: \"3f72be6f-6647-49dc-a7dc-2d722f870fed\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd"
Oct 30 21:38:30.254751 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.254691    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f72be6f-6647-49dc-a7dc-2d722f870fed-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd\" (UID: \"3f72be6f-6647-49dc-a7dc-2d722f870fed\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd"
Oct 30 21:38:30.339479 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.339450    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cgs5\" (UniqueName: \"kubernetes.io/projected/3f72be6f-6647-49dc-a7dc-2d722f870fed-kube-api-access-6cgs5\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd\" (UID: \"3f72be6f-6647-49dc-a7dc-2d722f870fed\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd"
Oct 30 21:38:30.348100 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.348061    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd"
Oct 30 21:38:30.668422 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.668368    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c" event={"ID":"4d4af674-8580-417f-b34d-6d593c9f05e3","Type":"ContainerDied","Data":"108fea0a56b04b31cf077db1bda96b4ec7729f5355a72725294ea8276c20c1ae"}
Oct 30 21:38:30.668422 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.668427    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="108fea0a56b04b31cf077db1bda96b4ec7729f5355a72725294ea8276c20c1ae"
Oct 30 21:38:30.668706 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.668424    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644jwb7c"
Oct 30 21:38:30.867799 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:30.867774    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd"]
Oct 30 21:38:30.870591 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:38:30.870554    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f72be6f_6647_49dc_a7dc_2d722f870fed.slice/crio-c47b8970c28d8ccd7fe4c443892bb9743e98d418f8f90025bcd508f0a98b680f WatchSource:0}: Error finding container c47b8970c28d8ccd7fe4c443892bb9743e98d418f8f90025bcd508f0a98b680f: Status 404 returned error can't find the container with id c47b8970c28d8ccd7fe4c443892bb9743e98d418f8f90025bcd508f0a98b680f
Oct 30 21:38:31.675213 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:31.675173    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd" event={"ID":"3f72be6f-6647-49dc-a7dc-2d722f870fed","Type":"ContainerStarted","Data":"c47b8970c28d8ccd7fe4c443892bb9743e98d418f8f90025bcd508f0a98b680f"}
Oct 30 21:38:31.922326 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:31.922284    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:32.134142 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:32.134046    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:32.674674 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:32.674638    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:32.839859 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:32.839813    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:33.295928 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:33.295890    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:33.464798 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:33.464760    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:33.627573 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:33.627492    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"
Oct 30 21:38:34.632974 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:34.632945    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-xx2qj"
Oct 30 21:38:35.696437 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:35.696399    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq" event={"ID":"54e53152-2818-40cb-a054-c313ce649d24","Type":"ContainerStarted","Data":"9d25676ed8bbfce2e9e0c8d223c27baa4c640a99d93f215a81544d30b5fbe42c"}
Oct 30 21:38:35.698070 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:35.698046    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd" event={"ID":"3f72be6f-6647-49dc-a7dc-2d722f870fed","Type":"ContainerStarted","Data":"7b188fc682db920ab8f28ae2c2a893892afa86fbee4cf7928934abf8f0265293"}
Oct 30 21:38:35.821915 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:35.821839    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-q66hq" podStartSLOduration=1.807814922 podStartE2EDuration="9.821820428s" totalImagesPullingTime="8.014005506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:35.821799001 +0000 UTC m=+258.493702937" watchObservedRunningTime="2026-10-30 21:38:35.821820428 +0000 UTC m=+258.493724364"
Oct 30 21:38:35.940671 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:35.940617    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-m9ngd" podStartSLOduration=2.6947053 podStartE2EDuration="6.94060219s" totalImagesPullingTime="4.24589689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:35.938769009 +0000 UTC m=+258.610672944" watchObservedRunningTime="2026-10-30 21:38:35.94060219 +0000 UTC m=+258.612506125"
Oct 30 21:38:36.644361 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:36.644327    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7c746f769b-f5lgh"
Oct 30 21:38:38.043727 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:38.043690    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:38.313867 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:38.313778    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:38.578671 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:38.578576    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl"]
Oct 30 21:38:38.581488 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:38.581465    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl"
Oct 30 21:38:38.682818 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:38.682781    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 21:38:38.682818 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:38.682814    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-m6cth\""
Oct 30 21:38:38.683035 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:38.683024    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 21:38:38.684488 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:38.684466    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl"]
Oct 30 21:38:38.734034 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:38.734001    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52rbq\" (UniqueName: \"kubernetes.io/projected/b0c53a94-60ca-42f6-9d28-fbcc2fdf135e-kube-api-access-52rbq\") pod \"cert-manager-webhook-5dbbbf75cc-9kxgl\" (UID: \"b0c53a94-60ca-42f6-9d28-fbcc2fdf135e\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl"
Oct 30 21:38:38.734215 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:38.734045    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b0c53a94-60ca-42f6-9d28-fbcc2fdf135e-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-9kxgl\" (UID: \"b0c53a94-60ca-42f6-9d28-fbcc2fdf135e\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl"
Oct 30 21:38:38.834870 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:38.834781    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b0c53a94-60ca-42f6-9d28-fbcc2fdf135e-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-9kxgl\" (UID: \"b0c53a94-60ca-42f6-9d28-fbcc2fdf135e\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl"
Oct 30 21:38:38.835017 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:38.834884    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52rbq\" (UniqueName: \"kubernetes.io/projected/b0c53a94-60ca-42f6-9d28-fbcc2fdf135e-kube-api-access-52rbq\") pod \"cert-manager-webhook-5dbbbf75cc-9kxgl\" (UID: \"b0c53a94-60ca-42f6-9d28-fbcc2fdf135e\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl"
Oct 30 21:38:38.904982 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:38.904950    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52rbq\" (UniqueName: \"kubernetes.io/projected/b0c53a94-60ca-42f6-9d28-fbcc2fdf135e-kube-api-access-52rbq\") pod \"cert-manager-webhook-5dbbbf75cc-9kxgl\" (UID: \"b0c53a94-60ca-42f6-9d28-fbcc2fdf135e\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl"
Oct 30 21:38:38.905459 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:38.905439    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b0c53a94-60ca-42f6-9d28-fbcc2fdf135e-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-9kxgl\" (UID: \"b0c53a94-60ca-42f6-9d28-fbcc2fdf135e\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl"
Oct 30 21:38:39.192397 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:39.192363    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl"
Oct 30 21:38:39.563541 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:39.563516    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl"]
Oct 30 21:38:39.565770 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:38:39.565741    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0c53a94_60ca_42f6_9d28_fbcc2fdf135e.slice/crio-97123029ab1022e15095bf9c3d77bf8fdfb5b6e2502594cdc4a18966fbdb23c5 WatchSource:0}: Error finding container 97123029ab1022e15095bf9c3d77bf8fdfb5b6e2502594cdc4a18966fbdb23c5: Status 404 returned error can't find the container with id 97123029ab1022e15095bf9c3d77bf8fdfb5b6e2502594cdc4a18966fbdb23c5
Oct 30 21:38:39.718084 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:39.718047    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl" event={"ID":"b0c53a94-60ca-42f6-9d28-fbcc2fdf135e","Type":"ContainerStarted","Data":"97123029ab1022e15095bf9c3d77bf8fdfb5b6e2502594cdc4a18966fbdb23c5"}
Oct 30 21:38:40.378821 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:40.378784    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:40.910714 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:40.910674    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:41.128478 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:41.128441    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:41.493911 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:41.493869    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:41.702165 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:41.702130    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:43.902546 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:43.902516    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-rbjkl"]
Oct 30 21:38:43.905518 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:43.905497    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rbjkl"
Oct 30 21:38:43.961982 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:43.961946    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:38:43.972638 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:43.972606    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Oct 30 21:38:43.978222 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:43.978189    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59a2f6f1-916d-40fe-9e03-d44f5779b573-tmp\") pod \"jobset-operator-674cd59dc5-rbjkl\" (UID: \"59a2f6f1-916d-40fe-9e03-d44f5779b573\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rbjkl"
Oct 30 21:38:43.978586 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:43.978339    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd9j7\" (UniqueName: \"kubernetes.io/projected/59a2f6f1-916d-40fe-9e03-d44f5779b573-kube-api-access-xd9j7\") pod \"jobset-operator-674cd59dc5-rbjkl\" (UID: \"59a2f6f1-916d-40fe-9e03-d44f5779b573\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rbjkl"
Oct 30 21:38:44.011343 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:44.011310    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-rzffs\""
Oct 30 21:38:44.012850 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:44.012825    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-rbjkl"]
Oct 30 21:38:44.079517 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:44.079482    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59a2f6f1-916d-40fe-9e03-d44f5779b573-tmp\") pod \"jobset-operator-674cd59dc5-rbjkl\" (UID: \"59a2f6f1-916d-40fe-9e03-d44f5779b573\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rbjkl"
Oct 30 21:38:44.079719 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:44.079551    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd9j7\" (UniqueName: \"kubernetes.io/projected/59a2f6f1-916d-40fe-9e03-d44f5779b573-kube-api-access-xd9j7\") pod \"jobset-operator-674cd59dc5-rbjkl\" (UID: \"59a2f6f1-916d-40fe-9e03-d44f5779b573\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rbjkl"
Oct 30 21:38:44.079928 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:44.079902    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59a2f6f1-916d-40fe-9e03-d44f5779b573-tmp\") pod \"jobset-operator-674cd59dc5-rbjkl\" (UID: \"59a2f6f1-916d-40fe-9e03-d44f5779b573\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rbjkl"
Oct 30 21:38:44.135735 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:44.135663    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd9j7\" (UniqueName: \"kubernetes.io/projected/59a2f6f1-916d-40fe-9e03-d44f5779b573-kube-api-access-xd9j7\") pod \"jobset-operator-674cd59dc5-rbjkl\" (UID: \"59a2f6f1-916d-40fe-9e03-d44f5779b573\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rbjkl"
Oct 30 21:38:44.226168 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:44.226125    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rbjkl"
Oct 30 21:38:44.605521 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:44.605496    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-rbjkl"]
Oct 30 21:38:44.608025 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:38:44.607999    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59a2f6f1_916d_40fe_9e03_d44f5779b573.slice/crio-c91bae3aa3200a6bad5d74a86e77a77483ae3e4968aca24f14553f09c1f6b6a6 WatchSource:0}: Error finding container c91bae3aa3200a6bad5d74a86e77a77483ae3e4968aca24f14553f09c1f6b6a6: Status 404 returned error can't find the container with id c91bae3aa3200a6bad5d74a86e77a77483ae3e4968aca24f14553f09c1f6b6a6
Oct 30 21:38:44.744953 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:44.744919    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rbjkl" event={"ID":"59a2f6f1-916d-40fe-9e03-d44f5779b573","Type":"ContainerStarted","Data":"c91bae3aa3200a6bad5d74a86e77a77483ae3e4968aca24f14553f09c1f6b6a6"}
Oct 30 21:38:44.746410 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:44.746384    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl" event={"ID":"b0c53a94-60ca-42f6-9d28-fbcc2fdf135e","Type":"ContainerStarted","Data":"5384c576f725bda8861a5d00dde4815a707c9a73ebce3d924ca9058c57cb691e"}
Oct 30 21:38:44.746523 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:44.746502    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl"
Oct 30 21:38:44.876528 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:44.876416    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl" podStartSLOduration=2.577034964 podStartE2EDuration="6.876401913s" totalImagesPullingTime="4.299366949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:44.87538947 +0000 UTC m=+267.547293406" watchObservedRunningTime="2026-10-30 21:38:44.876401913 +0000 UTC m=+267.548305848"
Oct 30 21:38:47.075754 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:47.075718    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:47.270537 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:47.270503    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:47.663108 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:47.663074    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:47.847331 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:47.847292    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:48.360779 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:38:48.360736    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:38:48.783949 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:48.783908    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:50.424274 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:50.424221    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-rcnkf"]
Oct 30 21:38:50.427225 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:50.427192    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-rcnkf"
Oct 30 21:38:50.484342 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:50.484308    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-rcnkf"]
Oct 30 21:38:50.527172 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:50.527138    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aad794ea-7d57-4018-a6c2-8fb1c551da43-bound-sa-token\") pod \"cert-manager-597b95cb84-rcnkf\" (UID: \"aad794ea-7d57-4018-a6c2-8fb1c551da43\") " pod="cert-manager/cert-manager-597b95cb84-rcnkf"
Oct 30 21:38:50.527351 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:50.527270    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fthjl\" (UniqueName: \"kubernetes.io/projected/aad794ea-7d57-4018-a6c2-8fb1c551da43-kube-api-access-fthjl\") pod \"cert-manager-597b95cb84-rcnkf\" (UID: \"aad794ea-7d57-4018-a6c2-8fb1c551da43\") " pod="cert-manager/cert-manager-597b95cb84-rcnkf"
Oct 30 21:38:50.531048 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:50.531019    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-569vt\""
Oct 30 21:38:50.628560 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:50.628523    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fthjl\" (UniqueName: \"kubernetes.io/projected/aad794ea-7d57-4018-a6c2-8fb1c551da43-kube-api-access-fthjl\") pod \"cert-manager-597b95cb84-rcnkf\" (UID: \"aad794ea-7d57-4018-a6c2-8fb1c551da43\") " pod="cert-manager/cert-manager-597b95cb84-rcnkf"
Oct 30 21:38:50.628744 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:50.628595    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aad794ea-7d57-4018-a6c2-8fb1c551da43-bound-sa-token\") pod \"cert-manager-597b95cb84-rcnkf\" (UID: \"aad794ea-7d57-4018-a6c2-8fb1c551da43\") " pod="cert-manager/cert-manager-597b95cb84-rcnkf"
Oct 30 21:38:50.692076 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:50.692002    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aad794ea-7d57-4018-a6c2-8fb1c551da43-bound-sa-token\") pod \"cert-manager-597b95cb84-rcnkf\" (UID: \"aad794ea-7d57-4018-a6c2-8fb1c551da43\") " pod="cert-manager/cert-manager-597b95cb84-rcnkf"
Oct 30 21:38:50.692076 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:50.692042    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fthjl\" (UniqueName: \"kubernetes.io/projected/aad794ea-7d57-4018-a6c2-8fb1c551da43-kube-api-access-fthjl\") pod \"cert-manager-597b95cb84-rcnkf\" (UID: \"aad794ea-7d57-4018-a6c2-8fb1c551da43\") " pod="cert-manager/cert-manager-597b95cb84-rcnkf"
Oct 30 21:38:50.740169 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:50.740131    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-rcnkf"
Oct 30 21:38:50.753692 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:50.753667    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-9kxgl"
Oct 30 21:38:50.779072 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:50.779032    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rbjkl" event={"ID":"59a2f6f1-916d-40fe-9e03-d44f5779b573","Type":"ContainerStarted","Data":"90126bca4827884c303664202ba4325e9423205d8d0fe3b37679c82debcdc222"}
Oct 30 21:38:50.986327 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:50.986208    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rbjkl" podStartSLOduration=2.03283781 podStartE2EDuration="7.986192183s" totalImagesPullingTime="5.953354373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:50.98567355 +0000 UTC m=+273.657577486" watchObservedRunningTime="2026-10-30 21:38:50.986192183 +0000 UTC m=+273.658096118"
Oct 30 21:38:51.114528 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:51.114409    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-rcnkf"]
Oct 30 21:38:51.117257 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:38:51.117210    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaad794ea_7d57_4018_a6c2_8fb1c551da43.slice/crio-519c301a41e82aa0a6e6ba424c0cfc332528d3577606d145c6948bb974ea6792 WatchSource:0}: Error finding container 519c301a41e82aa0a6e6ba424c0cfc332528d3577606d145c6948bb974ea6792: Status 404 returned error can't find the container with id 519c301a41e82aa0a6e6ba424c0cfc332528d3577606d145c6948bb974ea6792
Oct 30 21:38:51.785102 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:51.785066    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-rcnkf" event={"ID":"aad794ea-7d57-4018-a6c2-8fb1c551da43","Type":"ContainerStarted","Data":"ddf09f7f34f09b12ec0fcefce3074575b9ed3db9e9fdf295e8fc594cdbc31d46"}
Oct 30 21:38:51.785624 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:51.785117    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-rcnkf" event={"ID":"aad794ea-7d57-4018-a6c2-8fb1c551da43","Type":"ContainerStarted","Data":"519c301a41e82aa0a6e6ba424c0cfc332528d3577606d145c6948bb974ea6792"}
Oct 30 21:38:51.897185 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:51.897121    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-rcnkf" podStartSLOduration=1.897104354 podStartE2EDuration="1.897104354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:51.896707738 +0000 UTC m=+274.568611674" watchObservedRunningTime="2026-10-30 21:38:51.897104354 +0000 UTC m=+274.569008282"
Oct 30 21:38:53.257246 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.257200    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:53.409486 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.409448    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"]
Oct 30 21:38:53.409762 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.409723    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv" podUID="b77cc961-5c28-4437-83fe-037e93fecdb6" containerName="manager" containerID="cri-o://d0eb92da15768d498391ebecf5e4f943ec2b941358a11c6113730d7bfcc65fef" gracePeriod=10
Oct 30 21:38:53.419258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.419183    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:53.689957 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.689918    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"
Oct 30 21:38:53.757741 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.757704    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b77cc961-5c28-4437-83fe-037e93fecdb6-kube-api-access-p9689\" (UniqueName: \"kubernetes.io/projected/b77cc961-5c28-4437-83fe-037e93fecdb6-kube-api-access-p9689\") pod \"b77cc961-5c28-4437-83fe-037e93fecdb6\" (UID: \"b77cc961-5c28-4437-83fe-037e93fecdb6\") "
Oct 30 21:38:53.757895 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.757748    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b77cc961-5c28-4437-83fe-037e93fecdb6-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b77cc961-5c28-4437-83fe-037e93fecdb6-extensions-socket-volume\") pod \"b77cc961-5c28-4437-83fe-037e93fecdb6\" (UID: \"b77cc961-5c28-4437-83fe-037e93fecdb6\") "
Oct 30 21:38:53.758292 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.758257    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b77cc961-5c28-4437-83fe-037e93fecdb6-extensions-socket-volume" pod "b77cc961-5c28-4437-83fe-037e93fecdb6" (UID: "b77cc961-5c28-4437-83fe-037e93fecdb6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:38:53.759992 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.759966    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b77cc961-5c28-4437-83fe-037e93fecdb6-kube-api-access-p9689" pod "b77cc961-5c28-4437-83fe-037e93fecdb6" (UID: "b77cc961-5c28-4437-83fe-037e93fecdb6"). InnerVolumeSpecName "kube-api-access-p9689". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:53.796053 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.796023    5586 generic.go:356] "Generic (PLEG): container finished" podID="b77cc961-5c28-4437-83fe-037e93fecdb6" containerID="d0eb92da15768d498391ebecf5e4f943ec2b941358a11c6113730d7bfcc65fef" exitCode=0
Oct 30 21:38:53.796220 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.796069    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"
Oct 30 21:38:53.796220 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.796110    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv" event={"ID":"b77cc961-5c28-4437-83fe-037e93fecdb6","Type":"ContainerDied","Data":"d0eb92da15768d498391ebecf5e4f943ec2b941358a11c6113730d7bfcc65fef"}
Oct 30 21:38:53.796220 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.796163    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv" event={"ID":"b77cc961-5c28-4437-83fe-037e93fecdb6","Type":"ContainerDied","Data":"a2c20225593a1104c4babe85369395ac7e31ad7e92bf970d354b52d26c13fdea"}
Oct 30 21:38:53.796220 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.796185    5586 scope.go:122] "RemoveContainer" containerID="d0eb92da15768d498391ebecf5e4f943ec2b941358a11c6113730d7bfcc65fef"
Oct 30 21:38:53.812088 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.812068    5586 scope.go:122] "RemoveContainer" containerID="d0eb92da15768d498391ebecf5e4f943ec2b941358a11c6113730d7bfcc65fef"
Oct 30 21:38:53.812442 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:38:53.812424    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0eb92da15768d498391ebecf5e4f943ec2b941358a11c6113730d7bfcc65fef\": container with ID starting with d0eb92da15768d498391ebecf5e4f943ec2b941358a11c6113730d7bfcc65fef not found: ID does not exist" containerID="d0eb92da15768d498391ebecf5e4f943ec2b941358a11c6113730d7bfcc65fef"
Oct 30 21:38:53.812487 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.812468    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0eb92da15768d498391ebecf5e4f943ec2b941358a11c6113730d7bfcc65fef"} err="failed to get container status \"d0eb92da15768d498391ebecf5e4f943ec2b941358a11c6113730d7bfcc65fef\": rpc error: code = NotFound desc = could not find container \"d0eb92da15768d498391ebecf5e4f943ec2b941358a11c6113730d7bfcc65fef\": container with ID starting with d0eb92da15768d498391ebecf5e4f943ec2b941358a11c6113730d7bfcc65fef not found: ID does not exist"
Oct 30 21:38:53.859028 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.858995    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9689\" (UniqueName: \"kubernetes.io/projected/b77cc961-5c28-4437-83fe-037e93fecdb6-kube-api-access-p9689\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:53.859028 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.859021    5586 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b77cc961-5c28-4437-83fe-037e93fecdb6-extensions-socket-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:38:53.965984 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.965948    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"]
Oct 30 21:38:53.987402 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:53.987364    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv"]
Oct 30 21:38:54.030479 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:54.030436    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:54.194294 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:54.194253    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:54.624738 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:54.624633    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6h2cv" podUID="b77cc961-5c28-4437-83fe-037e93fecdb6" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.23:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:38:55.445571 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:55.445539    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b77cc961-5c28-4437-83fe-037e93fecdb6" path="/var/lib/kubelet/pods/b77cc961-5c28-4437-83fe-037e93fecdb6/volumes"
Oct 30 21:38:58.653627 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:58.653536    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:58.877813 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:58.877779    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:59.865397 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:38:59.865359    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:00.029513 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:00.029473    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:01.061813 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:01.061772    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:01.261810 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:01.261766    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:07.251539 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:07.251503    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:07.448832 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:07.448795    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:13.266047 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:13.266007    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:13.423539 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:13.423496    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:13.917587 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:13.917547    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:14.060475 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:14.060441    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:16.656896 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:16.656861    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:16.812296 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:16.812254    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:17.004011 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:17.003898    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:17.468053 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:17.468025    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 21:39:17.470907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:17.470881    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 21:39:17.655228 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:17.655186    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:17.816945 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:17.816844    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:18.173040 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:18.173001    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:18.361012 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:39:18.360970    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:39:19.802839 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:19.802792    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:19.859363 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:19.859327    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k"]
Oct 30 21:39:19.859672 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:19.859660    5586 cpu_manager.go:414] "containerMap: removing container" podUID="b77cc961-5c28-4437-83fe-037e93fecdb6" containerName="manager"
Oct 30 21:39:19.859719 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:19.859674    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b77cc961-5c28-4437-83fe-037e93fecdb6" containerName="manager"
Oct 30 21:39:19.859754 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:19.859745    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="b77cc961-5c28-4437-83fe-037e93fecdb6" containerName="manager"
Oct 30 21:39:19.876786 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:19.876756    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k"
Oct 30 21:39:19.922471 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:19.922437    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k"]
Oct 30 21:39:19.989037 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:19.988994    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/344bab0c-773b-45e2-bf3d-aad88f636c34-tls-certs\") pod \"kube-auth-proxy-6c46d956fb-t9g2k\" (UID: \"344bab0c-773b-45e2-bf3d-aad88f636c34\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k"
Oct 30 21:39:19.989037 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:19.989040    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/344bab0c-773b-45e2-bf3d-aad88f636c34-tmp\") pod \"kube-auth-proxy-6c46d956fb-t9g2k\" (UID: \"344bab0c-773b-45e2-bf3d-aad88f636c34\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k"
Oct 30 21:39:19.989271 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:19.989158    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgmbt\" (UniqueName: \"kubernetes.io/projected/344bab0c-773b-45e2-bf3d-aad88f636c34-kube-api-access-rgmbt\") pod \"kube-auth-proxy-6c46d956fb-t9g2k\" (UID: \"344bab0c-773b-45e2-bf3d-aad88f636c34\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k"
Oct 30 21:39:20.002707 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.002675    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sg8hv\""
Oct 30 21:39:20.002860 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.002761    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 21:39:20.002860 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.002774    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 21:39:20.034894 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.034862    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:20.089901 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.089812    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/344bab0c-773b-45e2-bf3d-aad88f636c34-tls-certs\") pod \"kube-auth-proxy-6c46d956fb-t9g2k\" (UID: \"344bab0c-773b-45e2-bf3d-aad88f636c34\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k"
Oct 30 21:39:20.089901 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.089854    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/344bab0c-773b-45e2-bf3d-aad88f636c34-tmp\") pod \"kube-auth-proxy-6c46d956fb-t9g2k\" (UID: \"344bab0c-773b-45e2-bf3d-aad88f636c34\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k"
Oct 30 21:39:20.090094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.089931    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgmbt\" (UniqueName: \"kubernetes.io/projected/344bab0c-773b-45e2-bf3d-aad88f636c34-kube-api-access-rgmbt\") pod \"kube-auth-proxy-6c46d956fb-t9g2k\" (UID: \"344bab0c-773b-45e2-bf3d-aad88f636c34\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k"
Oct 30 21:39:20.092197 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.092171    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/344bab0c-773b-45e2-bf3d-aad88f636c34-tmp\") pod \"kube-auth-proxy-6c46d956fb-t9g2k\" (UID: \"344bab0c-773b-45e2-bf3d-aad88f636c34\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k"
Oct 30 21:39:20.092428 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.092411    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/344bab0c-773b-45e2-bf3d-aad88f636c34-tls-certs\") pod \"kube-auth-proxy-6c46d956fb-t9g2k\" (UID: \"344bab0c-773b-45e2-bf3d-aad88f636c34\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k"
Oct 30 21:39:20.147722 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.147690    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgmbt\" (UniqueName: \"kubernetes.io/projected/344bab0c-773b-45e2-bf3d-aad88f636c34-kube-api-access-rgmbt\") pod \"kube-auth-proxy-6c46d956fb-t9g2k\" (UID: \"344bab0c-773b-45e2-bf3d-aad88f636c34\") " pod="openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k"
Oct 30 21:39:20.188021 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.187987    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k"
Oct 30 21:39:20.206222 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.206187    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:20.521833 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.521798    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:20.559648 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.559616    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k"]
Oct 30 21:39:20.562963 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:39:20.562936    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod344bab0c_773b_45e2_bf3d_aad88f636c34.slice/crio-34c7ddd914fe4d008a60f2f6c3cd5d0d8f03e2210c2a5d6cae09f6ecb15907ba WatchSource:0}: Error finding container 34c7ddd914fe4d008a60f2f6c3cd5d0d8f03e2210c2a5d6cae09f6ecb15907ba: Status 404 returned error can't find the container with id 34c7ddd914fe4d008a60f2f6c3cd5d0d8f03e2210c2a5d6cae09f6ecb15907ba
Oct 30 21:39:20.931922 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.931891    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k" event={"ID":"344bab0c-773b-45e2-bf3d-aad88f636c34","Type":"ContainerStarted","Data":"34c7ddd914fe4d008a60f2f6c3cd5d0d8f03e2210c2a5d6cae09f6ecb15907ba"}
Oct 30 21:39:20.985227 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:20.985189    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:21.562617 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:21.562512    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:21.631228 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:21.631192    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"]
Oct 30 21:39:21.636094 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:21.636066    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"
Oct 30 21:39:21.694640 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:21.693915    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"]
Oct 30 21:39:21.694640 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:21.694116    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 21:39:21.705929 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:21.705844    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74zd5\" (UniqueName: \"kubernetes.io/projected/a61a92ef-cfe7-4841-8f82-6aae6ad66aac-kube-api-access-74zd5\") pod \"limitador-limitador-74dfcfb894-wwrzq\" (UID: \"a61a92ef-cfe7-4841-8f82-6aae6ad66aac\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"
Oct 30 21:39:21.706102 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:21.705964    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a61a92ef-cfe7-4841-8f82-6aae6ad66aac-config-file\") pod \"limitador-limitador-74dfcfb894-wwrzq\" (UID: \"a61a92ef-cfe7-4841-8f82-6aae6ad66aac\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"
Oct 30 21:39:21.806802 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:21.806762    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74zd5\" (UniqueName: \"kubernetes.io/projected/a61a92ef-cfe7-4841-8f82-6aae6ad66aac-kube-api-access-74zd5\") pod \"limitador-limitador-74dfcfb894-wwrzq\" (UID: \"a61a92ef-cfe7-4841-8f82-6aae6ad66aac\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"
Oct 30 21:39:21.806988 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:21.806873    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a61a92ef-cfe7-4841-8f82-6aae6ad66aac-config-file\") pod \"limitador-limitador-74dfcfb894-wwrzq\" (UID: \"a61a92ef-cfe7-4841-8f82-6aae6ad66aac\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"
Oct 30 21:39:21.807714 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:21.807686    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a61a92ef-cfe7-4841-8f82-6aae6ad66aac-config-file\") pod \"limitador-limitador-74dfcfb894-wwrzq\" (UID: \"a61a92ef-cfe7-4841-8f82-6aae6ad66aac\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"
Oct 30 21:39:21.865509 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:21.865434    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74zd5\" (UniqueName: \"kubernetes.io/projected/a61a92ef-cfe7-4841-8f82-6aae6ad66aac-kube-api-access-74zd5\") pod \"limitador-limitador-74dfcfb894-wwrzq\" (UID: \"a61a92ef-cfe7-4841-8f82-6aae6ad66aac\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"
Oct 30 21:39:21.872644 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:21.872597    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"]
Oct 30 21:39:21.918343 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:21.918305    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:21.966271 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:21.965394    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"
Oct 30 21:39:22.076671 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:22.076638    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:22.262636 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:22.262597    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:22.355549 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:22.354551    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"]
Oct 30 21:39:22.361056 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:39:22.361026    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda61a92ef_cfe7_4841_8f82_6aae6ad66aac.slice/crio-85219d8e1a24cf2081280a2755782b11ad8235941adb5202d160c6e0f98510e0 WatchSource:0}: Error finding container 85219d8e1a24cf2081280a2755782b11ad8235941adb5202d160c6e0f98510e0: Status 404 returned error can't find the container with id 85219d8e1a24cf2081280a2755782b11ad8235941adb5202d160c6e0f98510e0
Oct 30 21:39:22.415817 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:22.415783    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:22.661756 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:22.661706    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:22.816692 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:22.816648    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:22.945834 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:22.945755    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq" event={"ID":"a61a92ef-cfe7-4841-8f82-6aae6ad66aac","Type":"ContainerStarted","Data":"85219d8e1a24cf2081280a2755782b11ad8235941adb5202d160c6e0f98510e0"}
Oct 30 21:39:23.360181 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:23.360092    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:23.520738 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:23.520686    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:23.857067 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:23.857026    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:23.957048 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:23.957002    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k" event={"ID":"344bab0c-773b-45e2-bf3d-aad88f636c34","Type":"ContainerStarted","Data":"92675b5d045a1a2302d4bf2917521ff1fbef1baf1c0b60a6e1ecaba6ff0fefe0"}
Oct 30 21:39:24.012523 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:24.012283    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:24.076676 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:24.076618    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6c46d956fb-t9g2k" podStartSLOduration=2.143803387 podStartE2EDuration="5.076604527s" totalImagesPullingTime="2.93280114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:24.07434691 +0000 UTC m=+306.746250846" watchObservedRunningTime="2026-10-30 21:39:24.076604527 +0000 UTC m=+306.748508461"
Oct 30 21:39:24.567654 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:24.567611    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:24.744922 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:24.744878    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:24.923193 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:24.923140    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:24.963888 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:24.963849    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq" event={"ID":"a61a92ef-cfe7-4841-8f82-6aae6ad66aac","Type":"ContainerStarted","Data":"5eb1fac46debdf47d0c57161fc76823ab0c179ce6070397aa9dfcc054397a7c2"}
Oct 30 21:39:24.964088 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:24.964070    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"
Oct 30 21:39:25.091605 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:25.091497    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq" podStartSLOduration=2.069156551 podStartE2EDuration="4.091482572s" totalImagesPullingTime="2.022326021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:25.089073192 +0000 UTC m=+307.760977126" watchObservedRunningTime="2026-10-30 21:39:25.091482572 +0000 UTC m=+307.763386506"
Oct 30 21:39:26.223184 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.223148    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"]
Oct 30 21:39:26.228371 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.228347    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:26.284090 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.284059    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Oct 30 21:39:26.290074 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.290041    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"]
Oct 30 21:39:26.290213 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.290095    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-cmq5g\""
Oct 30 21:39:26.290307 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.290291    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Oct 30 21:39:26.290307 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.290303    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Oct 30 21:39:26.350491 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.350445    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrcqk\" (UniqueName: \"kubernetes.io/projected/ac240b79-e71e-4e0d-998f-0fd259dcdb00-kube-api-access-xrcqk\") pod \"jobset-controller-manager-f68c9fb99-tr6lf\" (UID: \"ac240b79-e71e-4e0d-998f-0fd259dcdb00\") " pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:26.350491 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.350499    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ac240b79-e71e-4e0d-998f-0fd259dcdb00-manager-config\") pod \"jobset-controller-manager-f68c9fb99-tr6lf\" (UID: \"ac240b79-e71e-4e0d-998f-0fd259dcdb00\") " pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:26.350701 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.350669    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac240b79-e71e-4e0d-998f-0fd259dcdb00-metrics-certs\") pod \"jobset-controller-manager-f68c9fb99-tr6lf\" (UID: \"ac240b79-e71e-4e0d-998f-0fd259dcdb00\") " pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:26.350808 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.350788    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac240b79-e71e-4e0d-998f-0fd259dcdb00-cert\") pod \"jobset-controller-manager-f68c9fb99-tr6lf\" (UID: \"ac240b79-e71e-4e0d-998f-0fd259dcdb00\") " pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:26.451378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.451335    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac240b79-e71e-4e0d-998f-0fd259dcdb00-metrics-certs\") pod \"jobset-controller-manager-f68c9fb99-tr6lf\" (UID: \"ac240b79-e71e-4e0d-998f-0fd259dcdb00\") " pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:26.451558 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.451487    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac240b79-e71e-4e0d-998f-0fd259dcdb00-cert\") pod \"jobset-controller-manager-f68c9fb99-tr6lf\" (UID: \"ac240b79-e71e-4e0d-998f-0fd259dcdb00\") " pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:26.451558 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.451510    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrcqk\" (UniqueName: \"kubernetes.io/projected/ac240b79-e71e-4e0d-998f-0fd259dcdb00-kube-api-access-xrcqk\") pod \"jobset-controller-manager-f68c9fb99-tr6lf\" (UID: \"ac240b79-e71e-4e0d-998f-0fd259dcdb00\") " pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:26.451558 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.451538    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ac240b79-e71e-4e0d-998f-0fd259dcdb00-manager-config\") pod \"jobset-controller-manager-f68c9fb99-tr6lf\" (UID: \"ac240b79-e71e-4e0d-998f-0fd259dcdb00\") " pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:26.452219 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.452194    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ac240b79-e71e-4e0d-998f-0fd259dcdb00-manager-config\") pod \"jobset-controller-manager-f68c9fb99-tr6lf\" (UID: \"ac240b79-e71e-4e0d-998f-0fd259dcdb00\") " pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:26.453938 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.453912    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac240b79-e71e-4e0d-998f-0fd259dcdb00-metrics-certs\") pod \"jobset-controller-manager-f68c9fb99-tr6lf\" (UID: \"ac240b79-e71e-4e0d-998f-0fd259dcdb00\") " pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:26.454025 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.453937    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac240b79-e71e-4e0d-998f-0fd259dcdb00-cert\") pod \"jobset-controller-manager-f68c9fb99-tr6lf\" (UID: \"ac240b79-e71e-4e0d-998f-0fd259dcdb00\") " pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:26.509594 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.509513    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrcqk\" (UniqueName: \"kubernetes.io/projected/ac240b79-e71e-4e0d-998f-0fd259dcdb00-kube-api-access-xrcqk\") pod \"jobset-controller-manager-f68c9fb99-tr6lf\" (UID: \"ac240b79-e71e-4e0d-998f-0fd259dcdb00\") " pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:26.541164 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.541126    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:26.926373 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.926309    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"]
Oct 30 21:39:26.929013 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:39:26.928983    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac240b79_e71e_4e0d_998f_0fd259dcdb00.slice/crio-512e843fc3becb18a21e5957e55d101fc88eee33347bb8c30001ea9d52b3e8a4 WatchSource:0}: Error finding container 512e843fc3becb18a21e5957e55d101fc88eee33347bb8c30001ea9d52b3e8a4: Status 404 returned error can't find the container with id 512e843fc3becb18a21e5957e55d101fc88eee33347bb8c30001ea9d52b3e8a4
Oct 30 21:39:26.979849 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:26.979806    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf" event={"ID":"ac240b79-e71e-4e0d-998f-0fd259dcdb00","Type":"ContainerStarted","Data":"512e843fc3becb18a21e5957e55d101fc88eee33347bb8c30001ea9d52b3e8a4"}
Oct 30 21:39:28.041063 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:28.041022    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:30.279758 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:30.279692    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:30.432361 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:30.432324    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:32.014859 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:32.014823    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf" event={"ID":"ac240b79-e71e-4e0d-998f-0fd259dcdb00","Type":"ContainerStarted","Data":"66f0f481e7e7a1383b6c92836967327feb06b5bc0a63ba825cd55155904f98e4"}
Oct 30 21:39:32.015275 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:32.014947    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:32.136135 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:32.136080    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf" podStartSLOduration=2.0195317 podStartE2EDuration="6.136064241s" totalImagesPullingTime="4.116532541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:32.13547941 +0000 UTC m=+314.807383326" watchObservedRunningTime="2026-10-30 21:39:32.136064241 +0000 UTC m=+314.807968176"
Oct 30 21:39:35.970045 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:35.970006    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"
Oct 30 21:39:36.161168 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:36.161125    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:36.373143 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:36.373052    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:39.081202 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:39.081160    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:39.239511 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:39.239458    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:40.563873 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:40.563833    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:41.225454 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:41.225418    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"]
Oct 30 21:39:41.225652 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:41.225629    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq" podUID="a61a92ef-cfe7-4841-8f82-6aae6ad66aac" containerName="limitador" containerID="cri-o://5eb1fac46debdf47d0c57161fc76823ab0c179ce6070397aa9dfcc054397a7c2" gracePeriod=30
Oct 30 21:39:41.766695 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:41.766670    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"
Oct 30 21:39:41.793203 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:41.793107    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a61a92ef-cfe7-4841-8f82-6aae6ad66aac-config-file\" (UniqueName: \"kubernetes.io/configmap/a61a92ef-cfe7-4841-8f82-6aae6ad66aac-config-file\") pod \"a61a92ef-cfe7-4841-8f82-6aae6ad66aac\" (UID: \"a61a92ef-cfe7-4841-8f82-6aae6ad66aac\") "
Oct 30 21:39:41.793376 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:41.793214    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a61a92ef-cfe7-4841-8f82-6aae6ad66aac-kube-api-access-74zd5\" (UniqueName: \"kubernetes.io/projected/a61a92ef-cfe7-4841-8f82-6aae6ad66aac-kube-api-access-74zd5\") pod \"a61a92ef-cfe7-4841-8f82-6aae6ad66aac\" (UID: \"a61a92ef-cfe7-4841-8f82-6aae6ad66aac\") "
Oct 30 21:39:41.793576 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:41.793549    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a61a92ef-cfe7-4841-8f82-6aae6ad66aac-config-file" pod "a61a92ef-cfe7-4841-8f82-6aae6ad66aac" (UID: "a61a92ef-cfe7-4841-8f82-6aae6ad66aac"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:39:41.795382 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:41.795361    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a61a92ef-cfe7-4841-8f82-6aae6ad66aac-kube-api-access-74zd5" pod "a61a92ef-cfe7-4841-8f82-6aae6ad66aac" (UID: "a61a92ef-cfe7-4841-8f82-6aae6ad66aac"). InnerVolumeSpecName "kube-api-access-74zd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:39:41.894865 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:41.894825    5586 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a61a92ef-cfe7-4841-8f82-6aae6ad66aac-config-file\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:39:41.894865 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:41.894862    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74zd5\" (UniqueName: \"kubernetes.io/projected/a61a92ef-cfe7-4841-8f82-6aae6ad66aac-kube-api-access-74zd5\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:39:42.073702 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:42.073607    5586 generic.go:356] "Generic (PLEG): container finished" podID="ac240b79-e71e-4e0d-998f-0fd259dcdb00" containerID="66f0f481e7e7a1383b6c92836967327feb06b5bc0a63ba825cd55155904f98e4" exitCode=1
Oct 30 21:39:42.073702 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:42.073689    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf" event={"ID":"ac240b79-e71e-4e0d-998f-0fd259dcdb00","Type":"ContainerDied","Data":"66f0f481e7e7a1383b6c92836967327feb06b5bc0a63ba825cd55155904f98e4"}
Oct 30 21:39:42.074194 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:42.074172    5586 scope.go:122] "RemoveContainer" containerID="66f0f481e7e7a1383b6c92836967327feb06b5bc0a63ba825cd55155904f98e4"
Oct 30 21:39:42.075368 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:42.075342    5586 generic.go:356] "Generic (PLEG): container finished" podID="a61a92ef-cfe7-4841-8f82-6aae6ad66aac" containerID="5eb1fac46debdf47d0c57161fc76823ab0c179ce6070397aa9dfcc054397a7c2" exitCode=0
Oct 30 21:39:42.075487 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:42.075395    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"
Oct 30 21:39:42.075487 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:42.075427    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq" event={"ID":"a61a92ef-cfe7-4841-8f82-6aae6ad66aac","Type":"ContainerDied","Data":"5eb1fac46debdf47d0c57161fc76823ab0c179ce6070397aa9dfcc054397a7c2"}
Oct 30 21:39:42.075487 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:42.075483    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-wwrzq" event={"ID":"a61a92ef-cfe7-4841-8f82-6aae6ad66aac","Type":"ContainerDied","Data":"85219d8e1a24cf2081280a2755782b11ad8235941adb5202d160c6e0f98510e0"}
Oct 30 21:39:42.075647 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:42.075514    5586 scope.go:122] "RemoveContainer" containerID="5eb1fac46debdf47d0c57161fc76823ab0c179ce6070397aa9dfcc054397a7c2"
Oct 30 21:39:42.091765 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:42.091737    5586 scope.go:122] "RemoveContainer" containerID="5eb1fac46debdf47d0c57161fc76823ab0c179ce6070397aa9dfcc054397a7c2"
Oct 30 21:39:42.092177 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:39:42.092148    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eb1fac46debdf47d0c57161fc76823ab0c179ce6070397aa9dfcc054397a7c2\": container with ID starting with 5eb1fac46debdf47d0c57161fc76823ab0c179ce6070397aa9dfcc054397a7c2 not found: ID does not exist" containerID="5eb1fac46debdf47d0c57161fc76823ab0c179ce6070397aa9dfcc054397a7c2"
Oct 30 21:39:42.092262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:42.092196    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eb1fac46debdf47d0c57161fc76823ab0c179ce6070397aa9dfcc054397a7c2"} err="failed to get container status \"5eb1fac46debdf47d0c57161fc76823ab0c179ce6070397aa9dfcc054397a7c2\": rpc error: code = NotFound desc = could not find container \"5eb1fac46debdf47d0c57161fc76823ab0c179ce6070397aa9dfcc054397a7c2\": container with ID starting with 5eb1fac46debdf47d0c57161fc76823ab0c179ce6070397aa9dfcc054397a7c2 not found: ID does not exist"
Oct 30 21:39:42.310532 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:42.310491    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"]
Oct 30 21:39:42.315306 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:42.315274    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-wwrzq"]
Oct 30 21:39:43.082261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:43.082214    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf" event={"ID":"ac240b79-e71e-4e0d-998f-0fd259dcdb00","Type":"ContainerStarted","Data":"1e36f29d1b09f20841750fc030cae115abfb187d48320b3cb1e043a697c00014"}
Oct 30 21:39:43.082704 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:43.082465    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:43.445699 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:43.445662    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a61a92ef-cfe7-4841-8f82-6aae6ad66aac" path="/var/lib/kubelet/pods/a61a92ef-cfe7-4841-8f82-6aae6ad66aac/volumes"
Oct 30 21:39:45.292419 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:45.292384    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:45.439165 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:45.439124    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:46.095817 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:46.095772    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:48.360325 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:39:48.360286    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:39:53.894187 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:53.894146    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:54.093375 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:54.093342    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-f68c9fb99-tr6lf"
Oct 30 21:39:59.148086 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:59.148048    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:59.306769 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:39:59.306730    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:16.852985 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:16.852950    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:17.053798 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:17.053754    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:17.242553 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:17.242513    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:18.360325 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:40:18.360281    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:40:21.189812 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:21.189766    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:21.588825 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:21.588742    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:24.517737 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:24.517699    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:24.703839 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:24.703801    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:38.732123 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:38.732032    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:38.887036 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:38.886990    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:39.045341 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.045226    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:39.201734 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.201698    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:39.354744 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.354654    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:39.524765 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.524728    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:39.701488 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.701449    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:39.792803 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.792767    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-mzp2h"]
Oct 30 21:40:39.793312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.793149    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a61a92ef-cfe7-4841-8f82-6aae6ad66aac" containerName="limitador"
Oct 30 21:40:39.793312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.793162    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a61a92ef-cfe7-4841-8f82-6aae6ad66aac" containerName="limitador"
Oct 30 21:40:39.793312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.793279    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="a61a92ef-cfe7-4841-8f82-6aae6ad66aac" containerName="limitador"
Oct 30 21:40:39.796562 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.796539    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-mzp2h"
Oct 30 21:40:39.853839 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.853802    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-mzp2h"]
Oct 30 21:40:39.859213 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.859174    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:39.896700 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.896666    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc9eae3c-37d2-4ae7-a9cd-92419715e4eb-tls-certs\") pod \"model-serving-api-75bd89fdfc-mzp2h\" (UID: \"bc9eae3c-37d2-4ae7-a9cd-92419715e4eb\") " pod="opendatahub/model-serving-api-75bd89fdfc-mzp2h"
Oct 30 21:40:39.896900 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.896713    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f279q\" (UniqueName: \"kubernetes.io/projected/bc9eae3c-37d2-4ae7-a9cd-92419715e4eb-kube-api-access-f279q\") pod \"model-serving-api-75bd89fdfc-mzp2h\" (UID: \"bc9eae3c-37d2-4ae7-a9cd-92419715e4eb\") " pod="opendatahub/model-serving-api-75bd89fdfc-mzp2h"
Oct 30 21:40:39.900733 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.900689    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Oct 30 21:40:39.900733 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.900689    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 21:40:39.900733 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.900716    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 21:40:39.900733 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.900688    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-j9psx\""
Oct 30 21:40:39.997915 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.997830    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc9eae3c-37d2-4ae7-a9cd-92419715e4eb-tls-certs\") pod \"model-serving-api-75bd89fdfc-mzp2h\" (UID: \"bc9eae3c-37d2-4ae7-a9cd-92419715e4eb\") " pod="opendatahub/model-serving-api-75bd89fdfc-mzp2h"
Oct 30 21:40:39.997915 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:39.997882    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f279q\" (UniqueName: \"kubernetes.io/projected/bc9eae3c-37d2-4ae7-a9cd-92419715e4eb-kube-api-access-f279q\") pod \"model-serving-api-75bd89fdfc-mzp2h\" (UID: \"bc9eae3c-37d2-4ae7-a9cd-92419715e4eb\") " pod="opendatahub/model-serving-api-75bd89fdfc-mzp2h"
Oct 30 21:40:40.000384 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:40.000364    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc9eae3c-37d2-4ae7-a9cd-92419715e4eb-tls-certs\") pod \"model-serving-api-75bd89fdfc-mzp2h\" (UID: \"bc9eae3c-37d2-4ae7-a9cd-92419715e4eb\") " pod="opendatahub/model-serving-api-75bd89fdfc-mzp2h"
Oct 30 21:40:40.055312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:40.055274    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f279q\" (UniqueName: \"kubernetes.io/projected/bc9eae3c-37d2-4ae7-a9cd-92419715e4eb-kube-api-access-f279q\") pod \"model-serving-api-75bd89fdfc-mzp2h\" (UID: \"bc9eae3c-37d2-4ae7-a9cd-92419715e4eb\") " pod="opendatahub/model-serving-api-75bd89fdfc-mzp2h"
Oct 30 21:40:40.109187 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:40.109150    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-mzp2h"
Oct 30 21:40:40.486499 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:40.486443    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-mzp2h"]
Oct 30 21:40:40.488891 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:40:40.488863    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc9eae3c_37d2_4ae7_a9cd_92419715e4eb.slice/crio-60e95972db7ea01f7286bfd46811d131cc5b3c57138384f48a9a4361ef548dd1 WatchSource:0}: Error finding container 60e95972db7ea01f7286bfd46811d131cc5b3c57138384f48a9a4361ef548dd1: Status 404 returned error can't find the container with id 60e95972db7ea01f7286bfd46811d131cc5b3c57138384f48a9a4361ef548dd1
Oct 30 21:40:40.947865 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:40.947832    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:41.368167 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:41.368077    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-mzp2h" event={"ID":"bc9eae3c-37d2-4ae7-a9cd-92419715e4eb","Type":"ContainerStarted","Data":"60e95972db7ea01f7286bfd46811d131cc5b3c57138384f48a9a4361ef548dd1"}
Oct 30 21:40:43.380830 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:43.380793    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-mzp2h" event={"ID":"bc9eae3c-37d2-4ae7-a9cd-92419715e4eb","Type":"ContainerStarted","Data":"076ac41d54b4b5052e04b3199918b213ec37fd2a7e8cae8be4baa61415df7c45"}
Oct 30 21:40:43.381263 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:43.380912    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-mzp2h"
Oct 30 21:40:43.496785 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:43.496727    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-mzp2h" podStartSLOduration=2.294653974 podStartE2EDuration="4.49670983s" totalImagesPullingTime="2.202055856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:43.49558167 +0000 UTC m=+386.167485606" watchObservedRunningTime="2026-10-30 21:40:43.49670983 +0000 UTC m=+386.168613766"
Oct 30 21:40:46.158649 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:46.158597    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:46.339625 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:46.339584    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:46.503590 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:46.503509    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:46.653375 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:46.653333    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:46.855289 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:46.855178    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:47.019894 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:47.019861    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:47.192593 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:47.192553    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:47.354075 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:47.354036    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:48.359818 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:40:48.359780    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:40:50.912359 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:50.912325    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:51.076277 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:51.076212    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:53.043815 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:53.043778    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:53.259446 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:53.259405    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:53.459387 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:53.459355    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:53.788076 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:53.787976    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:54.397320 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:40:54.397291    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-mzp2h"
Oct 30 21:41:04.083377 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:04.083338    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:04.343336 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:04.342947    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:09.511744 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:09.511708    5586 generic.go:356] "Generic (PLEG): container finished" podID="edcb968b-97be-45b0-8e26-39378e683676" containerID="50bcd58fc15c87b9bb68c433650fda8162f3d2e646620e8d85b527ddb89743ea" exitCode=0
Oct 30 21:41:09.512183 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:09.511779    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd" event={"ID":"edcb968b-97be-45b0-8e26-39378e683676","Type":"ContainerDied","Data":"50bcd58fc15c87b9bb68c433650fda8162f3d2e646620e8d85b527ddb89743ea"}
Oct 30 21:41:09.512183 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:09.512161    5586 scope.go:122] "RemoveContainer" containerID="50bcd58fc15c87b9bb68c433650fda8162f3d2e646620e8d85b527ddb89743ea"
Oct 30 21:41:09.513291 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:09.513262    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:41:09.513386 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:09.513328    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:41:09.513386 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:09.513371    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:41:10.518301 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:10.518263    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd" event={"ID":"edcb968b-97be-45b0-8e26-39378e683676","Type":"ContainerStarted","Data":"e7f27138cb98460c57f99bcb625ea6315ba52903bdf80005200ca8ff171edbc3"}
Oct 30 21:41:11.500333 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:11.500295    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:41:11.500577 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:11.500550    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Oct 30 21:41:11.500631 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:11.500608    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd" podUID="edcb968b-97be-45b0-8e26-39378e683676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Oct 30 21:41:12.500729 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:12.500684    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Oct 30 21:41:12.501143 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:12.500746    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd" podUID="edcb968b-97be-45b0-8e26-39378e683676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Oct 30 21:41:13.500225 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:13.500187    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Oct 30 21:41:13.500418 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:13.500287    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd" podUID="edcb968b-97be-45b0-8e26-39378e683676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Oct 30 21:41:14.500368 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:14.500330    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Oct 30 21:41:14.500774 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:14.500391    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd" podUID="edcb968b-97be-45b0-8e26-39378e683676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Oct 30 21:41:15.500389 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:15.500349    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Oct 30 21:41:15.500821 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:15.500408    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd" podUID="edcb968b-97be-45b0-8e26-39378e683676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Oct 30 21:41:16.504717 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:16.504687    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:41:16.505185 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:16.505101    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:41:16.505881 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:16.505863    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 21:41:17.506007 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:17.505965    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:18.359219 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:41:18.359181    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:41:20.097277 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:20.097219    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:20.252150 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:41:20.252107    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:48.361355 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:41:48.361311    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:42:17.848080 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:17.847983    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:18.360489 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:42:18.360450    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:42:23.052949 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:23.052912    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:23.217448 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:23.217399    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:34.259975 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:34.259920    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:34.413406 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:34.413369    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:40.309773 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:40.309725    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:43.869479 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:43.869430    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"]
Oct 30 21:42:43.873148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:43.873117    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:43.929135 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:43.929101    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c8745d6-d399-4ebf-ac9a-856cfae9f873-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:43.929446 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:43.929420    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c8745d6-d399-4ebf-ac9a-856cfae9f873-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:43.929623 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:43.929598    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzzg6\" (UniqueName: \"kubernetes.io/projected/4c8745d6-d399-4ebf-ac9a-856cfae9f873-kube-api-access-nzzg6\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:43.929739 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:43.929639    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c8745d6-d399-4ebf-ac9a-856cfae9f873-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:43.929800 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:43.929736    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c8745d6-d399-4ebf-ac9a-856cfae9f873-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:43.929864 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:43.929795    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c8745d6-d399-4ebf-ac9a-856cfae9f873-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:43.976207 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:43.976160    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 21:42:43.976479 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:43.976288    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4xmws\""
Oct 30 21:42:43.976479 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:43.976337    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 21:42:43.976479 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:43.976421    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Oct 30 21:42:43.979162 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:43.979125    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"]
Oct 30 21:42:43.985617 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:43.985590    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:44.031160 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.031115    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c8745d6-d399-4ebf-ac9a-856cfae9f873-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:44.031369 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.031171    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c8745d6-d399-4ebf-ac9a-856cfae9f873-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:44.031369 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.031269    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c8745d6-d399-4ebf-ac9a-856cfae9f873-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:44.031369 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.031313    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c8745d6-d399-4ebf-ac9a-856cfae9f873-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:44.031369 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.031350    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzzg6\" (UniqueName: \"kubernetes.io/projected/4c8745d6-d399-4ebf-ac9a-856cfae9f873-kube-api-access-nzzg6\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:44.031603 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.031379    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c8745d6-d399-4ebf-ac9a-856cfae9f873-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:44.031799 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.031771    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c8745d6-d399-4ebf-ac9a-856cfae9f873-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:44.031994 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.031883    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c8745d6-d399-4ebf-ac9a-856cfae9f873-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:44.032190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.032165    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c8745d6-d399-4ebf-ac9a-856cfae9f873-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:44.033871 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.033851    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c8745d6-d399-4ebf-ac9a-856cfae9f873-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:44.033986 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.033884    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c8745d6-d399-4ebf-ac9a-856cfae9f873-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:44.089119 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.089087    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzzg6\" (UniqueName: \"kubernetes.io/projected/4c8745d6-d399-4ebf-ac9a-856cfae9f873-kube-api-access-nzzg6\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-tksjb\" (UID: \"4c8745d6-d399-4ebf-ac9a-856cfae9f873\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:44.188615 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.188577    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:44.566110 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.566084    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"]
Oct 30 21:42:44.568884 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:42:44.568855    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c8745d6_d399_4ebf_ac9a_856cfae9f873.slice/crio-a1bcf7326d0ad9b465be052bc85f4b90e2586064522b285e2300d73ff81e8082 WatchSource:0}: Error finding container a1bcf7326d0ad9b465be052bc85f4b90e2586064522b285e2300d73ff81e8082: Status 404 returned error can't find the container with id a1bcf7326d0ad9b465be052bc85f4b90e2586064522b285e2300d73ff81e8082
Oct 30 21:42:44.570184 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:44.570164    5586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:42:45.005123 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:45.005078    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb" event={"ID":"4c8745d6-d399-4ebf-ac9a-856cfae9f873","Type":"ContainerStarted","Data":"a1bcf7326d0ad9b465be052bc85f4b90e2586064522b285e2300d73ff81e8082"}
Oct 30 21:42:47.018368 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:47.018327    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb" event={"ID":"4c8745d6-d399-4ebf-ac9a-856cfae9f873","Type":"ContainerStarted","Data":"57f2bfa8a704e2f072914411cdc0278638fbe77f0c517a055cbd5a28500651d3"}
Oct 30 21:42:47.136327 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:47.136272    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb" podStartSLOduration=2.626589085 podStartE2EDuration="4.136256918s" totalImagesPullingTime="1.509667833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:47.134608377 +0000 UTC m=+509.806512313" watchObservedRunningTime="2026-10-30 21:42:47.136256918 +0000 UTC m=+509.808160846"
Oct 30 21:42:47.188706 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:47.188670    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:47.395109 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:47.395023    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:48.360621 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:42:48.360579    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:42:53.162646 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.162607    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-2xrvx"]
Oct 30 21:42:53.166393 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.166371    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.223660 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.223625    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-2xrvx"]
Oct 30 21:42:53.267492 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.267461    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Oct 30 21:42:53.276959 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.276932    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:53.314862 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.314824    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/012a98b7-fc3b-43d2-8725-099a8eda00de-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.314862 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.314862    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/012a98b7-fc3b-43d2-8725-099a8eda00de-home\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.315074 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.314888    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlbr7\" (UniqueName: \"kubernetes.io/projected/012a98b7-fc3b-43d2-8725-099a8eda00de-kube-api-access-qlbr7\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.315074 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.314967    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/012a98b7-fc3b-43d2-8725-099a8eda00de-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.315074 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.314987    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/012a98b7-fc3b-43d2-8725-099a8eda00de-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.315074 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.315001    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/012a98b7-fc3b-43d2-8725-099a8eda00de-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.416157 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.416060    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/012a98b7-fc3b-43d2-8725-099a8eda00de-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.416157 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.416102    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/012a98b7-fc3b-43d2-8725-099a8eda00de-home\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.416157 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.416123    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlbr7\" (UniqueName: \"kubernetes.io/projected/012a98b7-fc3b-43d2-8725-099a8eda00de-kube-api-access-qlbr7\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.416497 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.416278    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/012a98b7-fc3b-43d2-8725-099a8eda00de-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.416497 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.416323    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/012a98b7-fc3b-43d2-8725-099a8eda00de-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.416497 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.416344    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/012a98b7-fc3b-43d2-8725-099a8eda00de-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.416653 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.416591    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/012a98b7-fc3b-43d2-8725-099a8eda00de-home\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.416717 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.416691    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/012a98b7-fc3b-43d2-8725-099a8eda00de-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.416764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.416697    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/012a98b7-fc3b-43d2-8725-099a8eda00de-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.418478 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.418449    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/012a98b7-fc3b-43d2-8725-099a8eda00de-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.418749 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.418730    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/012a98b7-fc3b-43d2-8725-099a8eda00de-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.472954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.472924    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlbr7\" (UniqueName: \"kubernetes.io/projected/012a98b7-fc3b-43d2-8725-099a8eda00de-kube-api-access-qlbr7\") pod \"echo-simulator-kserve-64b6f8f96c-2xrvx\" (UID: \"012a98b7-fc3b-43d2-8725-099a8eda00de\") " pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.478782 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.478753    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:53.852512 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:53.852483    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-2xrvx"]
Oct 30 21:42:53.854994 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:42:53.854964    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012a98b7_fc3b_43d2_8725_099a8eda00de.slice/crio-f054cc7ededb9e48b02210a7f02a810e531f665c92f14100738ffe300bda233f WatchSource:0}: Error finding container f054cc7ededb9e48b02210a7f02a810e531f665c92f14100738ffe300bda233f: Status 404 returned error can't find the container with id f054cc7ededb9e48b02210a7f02a810e531f665c92f14100738ffe300bda233f
Oct 30 21:42:54.055548 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:54.055505    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx" event={"ID":"012a98b7-fc3b-43d2-8725-099a8eda00de","Type":"ContainerStarted","Data":"f054cc7ededb9e48b02210a7f02a810e531f665c92f14100738ffe300bda233f"}
Oct 30 21:42:54.188969 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:54.188929    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:54.201500 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:54.201472    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:55.081197 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:55.081162    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb"
Oct 30 21:42:56.513972 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.513937    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"]
Oct 30 21:42:56.517388 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.517368    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.569469 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.569389    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Oct 30 21:42:56.620490 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.620451    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"]
Oct 30 21:42:56.647991 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.647957    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7cacb27-5f0a-4d31-b165-6031f3e9226e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.647991 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.647997    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26mkx\" (UniqueName: \"kubernetes.io/projected/e7cacb27-5f0a-4d31-b165-6031f3e9226e-kube-api-access-26mkx\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.648297 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.648021    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7cacb27-5f0a-4d31-b165-6031f3e9226e-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.648297 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.648123    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7cacb27-5f0a-4d31-b165-6031f3e9226e-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.648297 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.648195    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7cacb27-5f0a-4d31-b165-6031f3e9226e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.648297 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.648246    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7cacb27-5f0a-4d31-b165-6031f3e9226e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.656823 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.656724    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:56.748996 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.748895    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7cacb27-5f0a-4d31-b165-6031f3e9226e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.748996 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.748952    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7cacb27-5f0a-4d31-b165-6031f3e9226e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.749277 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.749017    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7cacb27-5f0a-4d31-b165-6031f3e9226e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.749277 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.749052    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26mkx\" (UniqueName: \"kubernetes.io/projected/e7cacb27-5f0a-4d31-b165-6031f3e9226e-kube-api-access-26mkx\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.749277 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.749076    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7cacb27-5f0a-4d31-b165-6031f3e9226e-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.749277 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.749109    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7cacb27-5f0a-4d31-b165-6031f3e9226e-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.749491 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.749468    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7cacb27-5f0a-4d31-b165-6031f3e9226e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.749546 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.749489    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7cacb27-5f0a-4d31-b165-6031f3e9226e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.749598 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.749582    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7cacb27-5f0a-4d31-b165-6031f3e9226e-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.751472 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.751448    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7cacb27-5f0a-4d31-b165-6031f3e9226e-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.751603 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.751548    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7cacb27-5f0a-4d31-b165-6031f3e9226e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.817800 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.817756    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26mkx\" (UniqueName: \"kubernetes.io/projected/e7cacb27-5f0a-4d31-b165-6031f3e9226e-kube-api-access-26mkx\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw\" (UID: \"e7cacb27-5f0a-4d31-b165-6031f3e9226e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:56.846688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:56.846630    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:42:57.077732 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:57.077647    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 21:42:57.077732 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:57.077696    5586 generic.go:356] "Generic (PLEG): container finished" podID="012a98b7-fc3b-43d2-8725-099a8eda00de" containerID="94733ce453e8989efaeacb876ca6c7b23ca0b3a7d311ba5a6a4146de845aecd7" exitCode=137
Oct 30 21:42:57.077969 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:57.077818    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx" event={"ID":"012a98b7-fc3b-43d2-8725-099a8eda00de","Type":"ContainerDied","Data":"94733ce453e8989efaeacb876ca6c7b23ca0b3a7d311ba5a6a4146de845aecd7"}
Oct 30 21:42:57.078073 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:57.078053    5586 scope.go:122] "RemoveContainer" containerID="94733ce453e8989efaeacb876ca6c7b23ca0b3a7d311ba5a6a4146de845aecd7"
Oct 30 21:42:57.230805 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:57.230769    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"]
Oct 30 21:42:57.234500 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:42:57.234461    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7cacb27_5f0a_4d31_b165_6031f3e9226e.slice/crio-a2d3c136e7603ba154957ae3d9e6b83801a622de63fa543cc5d196834dcb30e8 WatchSource:0}: Error finding container a2d3c136e7603ba154957ae3d9e6b83801a622de63fa543cc5d196834dcb30e8: Status 404 returned error can't find the container with id a2d3c136e7603ba154957ae3d9e6b83801a622de63fa543cc5d196834dcb30e8
Oct 30 21:42:57.479214 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:57.479179    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:58.083907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:58.083797    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw" event={"ID":"e7cacb27-5f0a-4d31-b165-6031f3e9226e","Type":"ContainerStarted","Data":"bf592a1a3bb3219e567439639f861c7dcf64793c55a39fb9aeb50960dbb2557a"}
Oct 30 21:42:58.083907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:58.083855    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw" event={"ID":"e7cacb27-5f0a-4d31-b165-6031f3e9226e","Type":"ContainerStarted","Data":"a2d3c136e7603ba154957ae3d9e6b83801a622de63fa543cc5d196834dcb30e8"}
Oct 30 21:42:58.085746 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:58.085726    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 21:42:58.085914 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:58.085856    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx" event={"ID":"012a98b7-fc3b-43d2-8725-099a8eda00de","Type":"ContainerStarted","Data":"b40f7484c632a322d92049cb440f6b80c58511de44572de4af7a698de655d02a"}
Oct 30 21:42:58.208019 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:58.207962    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw" podStartSLOduration=1.570723043 podStartE2EDuration="2.207946826s" totalImagesPullingTime="637.223783ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:58.2041292 +0000 UTC m=+520.876033132" watchObservedRunningTime="2026-10-30 21:42:58.207946826 +0000 UTC m=+520.879850759"
Oct 30 21:42:58.324066 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:58.323995    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx" podStartSLOduration=2.518575427 podStartE2EDuration="5.323977204s" totalImagesPullingTime="2.805401777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:58.323197857 +0000 UTC m=+520.995101792" watchObservedRunningTime="2026-10-30 21:42:58.323977204 +0000 UTC m=+520.995881137"
Oct 30 21:42:58.479368 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:58.479325    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:42:58.847035 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:42:58.846943    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:43:00.203543 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:00.203506    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:03.479943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:03.479895    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:43:03.492644 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:03.492614    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:43:03.856389 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:03.856290    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:04.129747 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:04.129664    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-2xrvx"
Oct 30 21:43:06.847675 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:06.847639    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:43:06.860261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:06.860220    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:43:06.919413 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:06.919377    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:07.147415 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:07.147333    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw"
Oct 30 21:43:10.835461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:10.835421    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"]
Oct 30 21:43:10.841617 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:10.841585    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:10.954740 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:10.954702    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"]
Oct 30 21:43:10.985760 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:10.985723    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:10.985938 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:10.985766    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:10.985938 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:10.985861    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:10.986020 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:10.985953    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:10.986020 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:10.986009    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhvp2\" (UniqueName: \"kubernetes.io/projected/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-kube-api-access-fhvp2\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:10.986108 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:10.986031    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:10.986108 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:10.986056    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:10.986196 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:10.986180    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:10.986270 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:10.986203    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.087496 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.087406    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.087496 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.087439    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.087496 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.087469    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.087496 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.087490    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.087834 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.087525    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.087834 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.087568    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.087834 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.087605    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhvp2\" (UniqueName: \"kubernetes.io/projected/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-kube-api-access-fhvp2\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.087834 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.087628    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.087834 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.087662    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.088087 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.087899    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.088087 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.087985    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.088197 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.088176    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.088338 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.088310    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.088881 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.088856    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.089884 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.089864    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.090137 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.090119    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.150538 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.150506    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.150709 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.150596    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhvp2\" (UniqueName: \"kubernetes.io/projected/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-kube-api-access-fhvp2\") pod \"maas-default-gateway-openshift-default-74854d8ccf-v4zqk\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.154559 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.154538    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:11.595638 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.595605    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"]
Oct 30 21:43:11.596221 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:43:11.596195    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podede8564d_fd7e_4cdf_94a2_7983f5d2e1cb.slice/crio-177bd343aeef6377790ee893e4e620c69a687aaf337ae8df0129672a7c9dee02 WatchSource:0}: Error finding container 177bd343aeef6377790ee893e4e620c69a687aaf337ae8df0129672a7c9dee02: Status 404 returned error can't find the container with id 177bd343aeef6377790ee893e4e620c69a687aaf337ae8df0129672a7c9dee02
Oct 30 21:43:11.598413 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.598381    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:43:11.598513 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.598447    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:43:11.598513 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:11.598479    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:43:12.164190 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:12.164148    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk" event={"ID":"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb","Type":"ContainerStarted","Data":"a07e6ab6c1da9515a0ff4fc6dc49ac1ecd2ec044c4f7ce3826e0cda645e9e9af"}
Oct 30 21:43:12.164625 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:12.164213    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk" event={"ID":"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb","Type":"ContainerStarted","Data":"177bd343aeef6377790ee893e4e620c69a687aaf337ae8df0129672a7c9dee02"}
Oct 30 21:43:12.286179 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:12.286120    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk" podStartSLOduration=2.286105001 podStartE2EDuration="2.286105001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:12.284708195 +0000 UTC m=+534.956612148" watchObservedRunningTime="2026-10-30 21:43:12.286105001 +0000 UTC m=+534.958008936"
Oct 30 21:43:13.154863 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:13.154822    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:13.156423 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:13.156390    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:43:13.156533 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:13.156478    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk" podUID="ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:43:14.154970 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:14.154932    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:43:14.155521 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:14.155004    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk" podUID="ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:43:15.155381 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:15.155339    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:43:15.155986 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:15.155947    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk" podUID="ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:43:16.156661 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:16.156622    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:43:16.157261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:16.157205    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk" podUID="ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:43:17.156122 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:17.155770    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:43:17.156122 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:17.155832    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk" podUID="ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:43:18.154927 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:18.154889    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:43:18.155363 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:18.154956    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk" podUID="ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:43:18.209145 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:18.209108    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:18.360018 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:43:18.359978    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:43:20.155533 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:20.155486    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:43:20.155965 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:20.155562    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk" podUID="ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:21.197426 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:21.197382    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-v4zqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:43:21.197926 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:21.197448    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk" podUID="ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:43:21.378786 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:21.378753    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:21.379150 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:21.379121    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:21.381283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:21.381266    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:43:23.901082 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:23.901048    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:29.530505 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:29.530416    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:35.504885 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.504846    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-685df8ccfd-7w6pg"]
Oct 30 21:43:35.508408 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.508387    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-685df8ccfd-7w6pg"
Oct 30 21:43:35.556725 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.556676    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvcxd\" (UniqueName: \"kubernetes.io/projected/f60cdd87-038a-4f42-a56d-c2d3f32911ef-kube-api-access-pvcxd\") pod \"authorino-685df8ccfd-7w6pg\" (UID: \"f60cdd87-038a-4f42-a56d-c2d3f32911ef\") " pod="kuadrant-system/authorino-685df8ccfd-7w6pg"
Oct 30 21:43:35.556725 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.556738    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f60cdd87-038a-4f42-a56d-c2d3f32911ef-tls-cert\") pod \"authorino-685df8ccfd-7w6pg\" (UID: \"f60cdd87-038a-4f42-a56d-c2d3f32911ef\") " pod="kuadrant-system/authorino-685df8ccfd-7w6pg"
Oct 30 21:43:35.562824 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.562771    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 21:43:35.565223 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.565197    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-685df8ccfd-7w6pg"]
Oct 30 21:43:35.612015 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.611978    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5956v\""
Oct 30 21:43:35.657653 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.657609    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvcxd\" (UniqueName: \"kubernetes.io/projected/f60cdd87-038a-4f42-a56d-c2d3f32911ef-kube-api-access-pvcxd\") pod \"authorino-685df8ccfd-7w6pg\" (UID: \"f60cdd87-038a-4f42-a56d-c2d3f32911ef\") " pod="kuadrant-system/authorino-685df8ccfd-7w6pg"
Oct 30 21:43:35.657653 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.657652    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f60cdd87-038a-4f42-a56d-c2d3f32911ef-tls-cert\") pod \"authorino-685df8ccfd-7w6pg\" (UID: \"f60cdd87-038a-4f42-a56d-c2d3f32911ef\") " pod="kuadrant-system/authorino-685df8ccfd-7w6pg"
Oct 30 21:43:35.660226 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.660195    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f60cdd87-038a-4f42-a56d-c2d3f32911ef-tls-cert\") pod \"authorino-685df8ccfd-7w6pg\" (UID: \"f60cdd87-038a-4f42-a56d-c2d3f32911ef\") " pod="kuadrant-system/authorino-685df8ccfd-7w6pg"
Oct 30 21:43:35.723865 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.723825    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-66dd76bf94-ktc28"]
Oct 30 21:43:35.725538 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.725511    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvcxd\" (UniqueName: \"kubernetes.io/projected/f60cdd87-038a-4f42-a56d-c2d3f32911ef-kube-api-access-pvcxd\") pod \"authorino-685df8ccfd-7w6pg\" (UID: \"f60cdd87-038a-4f42-a56d-c2d3f32911ef\") " pod="kuadrant-system/authorino-685df8ccfd-7w6pg"
Oct 30 21:43:35.727432 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.727417    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28"
Oct 30 21:43:35.735776 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.735744    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:35.783728 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.783635    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 21:43:35.783728 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.783643    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 30 21:43:35.783728 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.783711    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-6cdvr\""
Oct 30 21:43:35.786147 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.786120    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-66dd76bf94-ktc28"]
Oct 30 21:43:35.819928 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.819889    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-685df8ccfd-7w6pg"
Oct 30 21:43:35.859097 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.859060    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcxzz\" (UniqueName: \"kubernetes.io/projected/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8-kube-api-access-zcxzz\") pod \"ai-gateway-controller-66dd76bf94-ktc28\" (UID: \"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8\") " pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28"
Oct 30 21:43:35.859299 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.859274    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8-metrics-certs\") pod \"ai-gateway-controller-66dd76bf94-ktc28\" (UID: \"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8\") " pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28"
Oct 30 21:43:35.949914 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.949863    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:35.959840 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.959808    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcxzz\" (UniqueName: \"kubernetes.io/projected/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8-kube-api-access-zcxzz\") pod \"ai-gateway-controller-66dd76bf94-ktc28\" (UID: \"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8\") " pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28"
Oct 30 21:43:35.959993 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.959894    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8-metrics-certs\") pod \"ai-gateway-controller-66dd76bf94-ktc28\" (UID: \"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8\") " pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28"
Oct 30 21:43:35.962353 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:35.962324    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8-metrics-certs\") pod \"ai-gateway-controller-66dd76bf94-ktc28\" (UID: \"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8\") " pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28"
Oct 30 21:43:36.018955 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:36.018922    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcxzz\" (UniqueName: \"kubernetes.io/projected/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8-kube-api-access-zcxzz\") pod \"ai-gateway-controller-66dd76bf94-ktc28\" (UID: \"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8\") " pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28"
Oct 30 21:43:36.045386 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:36.045308    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28"
Oct 30 21:43:36.207012 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:36.206978    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-685df8ccfd-7w6pg"]
Oct 30 21:43:36.211354 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:43:36.211320    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf60cdd87_038a_4f42_a56d_c2d3f32911ef.slice/crio-09f976709e1542ad99bf799a12153ce9e3844136c90908e0755f41462fb57825 WatchSource:0}: Error finding container 09f976709e1542ad99bf799a12153ce9e3844136c90908e0755f41462fb57825: Status 404 returned error can't find the container with id 09f976709e1542ad99bf799a12153ce9e3844136c90908e0755f41462fb57825
Oct 30 21:43:36.293440 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:36.293403    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-685df8ccfd-7w6pg" event={"ID":"f60cdd87-038a-4f42-a56d-c2d3f32911ef","Type":"ContainerStarted","Data":"09f976709e1542ad99bf799a12153ce9e3844136c90908e0755f41462fb57825"}
Oct 30 21:43:36.437140 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:36.437109    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-66dd76bf94-ktc28"]
Oct 30 21:43:36.438726 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:43:36.438699    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd12bf7f_0fc3_43b6_a8fc_b68aafb37fb8.slice/crio-d7e75daf7c6fddb6f1c7c2dbf55da801ce59f5bf0b6ad280d469987a66d6f4d4 WatchSource:0}: Error finding container d7e75daf7c6fddb6f1c7c2dbf55da801ce59f5bf0b6ad280d469987a66d6f4d4: Status 404 returned error can't find the container with id d7e75daf7c6fddb6f1c7c2dbf55da801ce59f5bf0b6ad280d469987a66d6f4d4
Oct 30 21:43:37.299564 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:37.299525    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28" event={"ID":"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8","Type":"ContainerStarted","Data":"d7e75daf7c6fddb6f1c7c2dbf55da801ce59f5bf0b6ad280d469987a66d6f4d4"}
Oct 30 21:43:40.320805 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:40.320763    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-685df8ccfd-7w6pg" event={"ID":"f60cdd87-038a-4f42-a56d-c2d3f32911ef","Type":"ContainerStarted","Data":"bbb04481d13706641a0285b97df5a61671944267a615d81c734919c53e51815e"}
Oct 30 21:43:40.435244 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:40.435164    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-685df8ccfd-7w6pg" podStartSLOduration=2.170232995 podStartE2EDuration="5.435146757s" totalImagesPullingTime="3.264913762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:40.433596939 +0000 UTC m=+563.105500886" watchObservedRunningTime="2026-10-30 21:43:40.435146757 +0000 UTC m=+563.107050717"
Oct 30 21:43:42.340244 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:42.340127    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28" event={"ID":"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8","Type":"ContainerStarted","Data":"e1a57751f771cea96de756e47bf6aba5fb5949cecb79ed0f234d05dc68d1343e"}
Oct 30 21:43:42.340244 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:42.340190    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28"
Oct 30 21:43:42.465845 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:42.465788    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28" podStartSLOduration=1.8275407339999998 podStartE2EDuration="7.465773281s" totalImagesPullingTime="5.638232547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:42.464197726 +0000 UTC m=+565.136101661" watchObservedRunningTime="2026-10-30 21:43:42.465773281 +0000 UTC m=+565.137677217"
Oct 30 21:43:48.360650 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:43:48.360599    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:43:54.404397 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:43:54.404347    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28" podUID="cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.41:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:02.462652 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:02.462601    5586 generic.go:356] "Generic (PLEG): container finished" podID="cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8" containerID="e1a57751f771cea96de756e47bf6aba5fb5949cecb79ed0f234d05dc68d1343e" exitCode=1
Oct 30 21:44:02.463079 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:02.462677    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28" event={"ID":"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8","Type":"ContainerDied","Data":"e1a57751f771cea96de756e47bf6aba5fb5949cecb79ed0f234d05dc68d1343e"}
Oct 30 21:44:02.463124 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:02.463081    5586 scope.go:122] "RemoveContainer" containerID="e1a57751f771cea96de756e47bf6aba5fb5949cecb79ed0f234d05dc68d1343e"
Oct 30 21:44:03.469873 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:03.469837    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28" event={"ID":"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8","Type":"ContainerStarted","Data":"0eff629297a39d10920f5bc42966a5ec1328e262f4f98a284cc592ff0620b64a"}
Oct 30 21:44:03.470322 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:03.470042    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28"
Oct 30 21:44:06.108165 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.108126    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:06.324495 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.324447    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:06.497985 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.497951    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz"]
Oct 30 21:44:06.500472 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.500453    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz"
Oct 30 21:44:06.556640 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.556608    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 30 21:44:06.556836 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.556736    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-96fj2\""
Oct 30 21:44:06.562420 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.562388    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9h5w\" (UniqueName: \"kubernetes.io/projected/921e82fc-b274-48e4-ab0e-e8fbbeba9d2b-kube-api-access-m9h5w\") pod \"payload-pre-processing-7b98bb678c-l4jhz\" (UID: \"921e82fc-b274-48e4-ab0e-e8fbbeba9d2b\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz"
Oct 30 21:44:06.562551 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.562443    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/921e82fc-b274-48e4-ab0e-e8fbbeba9d2b-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-l4jhz\" (UID: \"921e82fc-b274-48e4-ab0e-e8fbbeba9d2b\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz"
Oct 30 21:44:06.608664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.608608    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz"]
Oct 30 21:44:06.659167 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.659121    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-8m8pr"]
Oct 30 21:44:06.661564 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.661549    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-8m8pr"
Oct 30 21:44:06.663440 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.663413    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9h5w\" (UniqueName: \"kubernetes.io/projected/921e82fc-b274-48e4-ab0e-e8fbbeba9d2b-kube-api-access-m9h5w\") pod \"payload-pre-processing-7b98bb678c-l4jhz\" (UID: \"921e82fc-b274-48e4-ab0e-e8fbbeba9d2b\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz"
Oct 30 21:44:06.663559 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.663502    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/921e82fc-b274-48e4-ab0e-e8fbbeba9d2b-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-l4jhz\" (UID: \"921e82fc-b274-48e4-ab0e-e8fbbeba9d2b\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz"
Oct 30 21:44:06.663559 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.663529    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbef39d1-31a9-4ca6-a5e0-dafaf19ed452-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-8m8pr\" (UID: \"cbef39d1-31a9-4ca6-a5e0-dafaf19ed452\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-8m8pr"
Oct 30 21:44:06.663559 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.663547    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrz8w\" (UniqueName: \"kubernetes.io/projected/cbef39d1-31a9-4ca6-a5e0-dafaf19ed452-kube-api-access-wrz8w\") pod \"payload-processing-55fbbbf9dc-8m8pr\" (UID: \"cbef39d1-31a9-4ca6-a5e0-dafaf19ed452\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-8m8pr"
Oct 30 21:44:06.664251 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.664215    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/921e82fc-b274-48e4-ab0e-e8fbbeba9d2b-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-l4jhz\" (UID: \"921e82fc-b274-48e4-ab0e-e8fbbeba9d2b\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz"
Oct 30 21:44:06.720884 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.720850    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-8m8pr"]
Oct 30 21:44:06.723245 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.723203    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9h5w\" (UniqueName: \"kubernetes.io/projected/921e82fc-b274-48e4-ab0e-e8fbbeba9d2b-kube-api-access-m9h5w\") pod \"payload-pre-processing-7b98bb678c-l4jhz\" (UID: \"921e82fc-b274-48e4-ab0e-e8fbbeba9d2b\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz"
Oct 30 21:44:06.764333 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.764204    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbef39d1-31a9-4ca6-a5e0-dafaf19ed452-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-8m8pr\" (UID: \"cbef39d1-31a9-4ca6-a5e0-dafaf19ed452\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-8m8pr"
Oct 30 21:44:06.764333 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.764291    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrz8w\" (UniqueName: \"kubernetes.io/projected/cbef39d1-31a9-4ca6-a5e0-dafaf19ed452-kube-api-access-wrz8w\") pod \"payload-processing-55fbbbf9dc-8m8pr\" (UID: \"cbef39d1-31a9-4ca6-a5e0-dafaf19ed452\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-8m8pr"
Oct 30 21:44:06.764949 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.764927    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbef39d1-31a9-4ca6-a5e0-dafaf19ed452-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-8m8pr\" (UID: \"cbef39d1-31a9-4ca6-a5e0-dafaf19ed452\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-8m8pr"
Oct 30 21:44:06.812045 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.812001    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz"
Oct 30 21:44:06.822109 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.822080    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrz8w\" (UniqueName: \"kubernetes.io/projected/cbef39d1-31a9-4ca6-a5e0-dafaf19ed452-kube-api-access-wrz8w\") pod \"payload-processing-55fbbbf9dc-8m8pr\" (UID: \"cbef39d1-31a9-4ca6-a5e0-dafaf19ed452\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-8m8pr"
Oct 30 21:44:06.974208 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:06.974169    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-8m8pr"
Oct 30 21:44:07.187446 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:07.187421    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz"]
Oct 30 21:44:07.189505 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:44:07.189481    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod921e82fc_b274_48e4_ab0e_e8fbbeba9d2b.slice/crio-6098438091b1d1da228827aa6112f83f08533843e46193db68d17da5b570937f WatchSource:0}: Error finding container 6098438091b1d1da228827aa6112f83f08533843e46193db68d17da5b570937f: Status 404 returned error can't find the container with id 6098438091b1d1da228827aa6112f83f08533843e46193db68d17da5b570937f
Oct 30 21:44:07.359027 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:07.359000    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-8m8pr"]
Oct 30 21:44:07.360818 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:44:07.360791    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbef39d1_31a9_4ca6_a5e0_dafaf19ed452.slice/crio-b44256d626dc76b00c059bcd815966090424a4cf44585d618661ce415475bf8e WatchSource:0}: Error finding container b44256d626dc76b00c059bcd815966090424a4cf44585d618661ce415475bf8e: Status 404 returned error can't find the container with id b44256d626dc76b00c059bcd815966090424a4cf44585d618661ce415475bf8e
Oct 30 21:44:07.494178 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:07.494140    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-8m8pr" event={"ID":"cbef39d1-31a9-4ca6-a5e0-dafaf19ed452","Type":"ContainerStarted","Data":"b44256d626dc76b00c059bcd815966090424a4cf44585d618661ce415475bf8e"}
Oct 30 21:44:07.495467 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:07.495437    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz" event={"ID":"921e82fc-b274-48e4-ab0e-e8fbbeba9d2b","Type":"ContainerStarted","Data":"6098438091b1d1da228827aa6112f83f08533843e46193db68d17da5b570937f"}
Oct 30 21:44:10.515436 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:10.515336    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz" event={"ID":"921e82fc-b274-48e4-ab0e-e8fbbeba9d2b","Type":"ContainerStarted","Data":"d4662bff2e2e43fed043a6f43e959aaedf11d4c695fbbb9aefe27175e322dc73"}
Oct 30 21:44:10.515893 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:10.515437    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz"
Oct 30 21:44:10.516951 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:10.516923    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-8m8pr" event={"ID":"cbef39d1-31a9-4ca6-a5e0-dafaf19ed452","Type":"ContainerStarted","Data":"881fb16067a0509a7fd5dbb66ba411dd356bc928324ea60cd93bc5f8eeb82940"}
Oct 30 21:44:10.517061 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:10.517049    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-8m8pr"
Oct 30 21:44:10.632036 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:10.631977    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz" podStartSLOduration=1.598005084 podStartE2EDuration="4.631959178s" totalImagesPullingTime="3.033954094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:10.629449608 +0000 UTC m=+593.301353543" watchObservedRunningTime="2026-10-30 21:44:10.631959178 +0000 UTC m=+593.303863114"
Oct 30 21:44:10.746551 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:10.746494    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-55fbbbf9dc-8m8pr" podStartSLOduration=1.849843559 podStartE2EDuration="4.746472121s" totalImagesPullingTime="2.896628562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:10.746153011 +0000 UTC m=+593.418056956" watchObservedRunningTime="2026-10-30 21:44:10.746472121 +0000 UTC m=+593.418376056"
Oct 30 21:44:14.477892 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:14.477852    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28"
Oct 30 21:44:14.833391 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:14.833303    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm"]
Oct 30 21:44:14.837151 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:14.837133    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm"
Oct 30 21:44:14.894927 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:14.894891    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm"]
Oct 30 21:44:14.934309 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:14.934273    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9b8v\" (UniqueName: \"kubernetes.io/projected/b2c4b8e3-b6db-4489-9344-6b3fa5bdd778-kube-api-access-h9b8v\") pod \"ai-gateway-controller-f74d5f7c9-mzjzm\" (UID: \"b2c4b8e3-b6db-4489-9344-6b3fa5bdd778\") " pod="opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm"
Oct 30 21:44:14.934495 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:14.934329    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2c4b8e3-b6db-4489-9344-6b3fa5bdd778-metrics-certs\") pod \"ai-gateway-controller-f74d5f7c9-mzjzm\" (UID: \"b2c4b8e3-b6db-4489-9344-6b3fa5bdd778\") " pod="opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm"
Oct 30 21:44:15.035056 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:15.035015    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9b8v\" (UniqueName: \"kubernetes.io/projected/b2c4b8e3-b6db-4489-9344-6b3fa5bdd778-kube-api-access-h9b8v\") pod \"ai-gateway-controller-f74d5f7c9-mzjzm\" (UID: \"b2c4b8e3-b6db-4489-9344-6b3fa5bdd778\") " pod="opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm"
Oct 30 21:44:15.035267 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:15.035081    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2c4b8e3-b6db-4489-9344-6b3fa5bdd778-metrics-certs\") pod \"ai-gateway-controller-f74d5f7c9-mzjzm\" (UID: \"b2c4b8e3-b6db-4489-9344-6b3fa5bdd778\") " pod="opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm"
Oct 30 21:44:15.037571 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:15.037548    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2c4b8e3-b6db-4489-9344-6b3fa5bdd778-metrics-certs\") pod \"ai-gateway-controller-f74d5f7c9-mzjzm\" (UID: \"b2c4b8e3-b6db-4489-9344-6b3fa5bdd778\") " pod="opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm"
Oct 30 21:44:15.094598 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:15.094521    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9b8v\" (UniqueName: \"kubernetes.io/projected/b2c4b8e3-b6db-4489-9344-6b3fa5bdd778-kube-api-access-h9b8v\") pod \"ai-gateway-controller-f74d5f7c9-mzjzm\" (UID: \"b2c4b8e3-b6db-4489-9344-6b3fa5bdd778\") " pod="opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm"
Oct 30 21:44:15.150458 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:15.150428    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm"
Oct 30 21:44:15.527625 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:15.527599    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm"]
Oct 30 21:44:15.530107 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:44:15.530077    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2c4b8e3_b6db_4489_9344_6b3fa5bdd778.slice/crio-a1fb729fa28ccee36cfa09ca7329dd591084d5a1b41f79573afded805858774b WatchSource:0}: Error finding container a1fb729fa28ccee36cfa09ca7329dd591084d5a1b41f79573afded805858774b: Status 404 returned error can't find the container with id a1fb729fa28ccee36cfa09ca7329dd591084d5a1b41f79573afded805858774b
Oct 30 21:44:15.551528 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:15.551492    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm" event={"ID":"b2c4b8e3-b6db-4489-9344-6b3fa5bdd778","Type":"ContainerStarted","Data":"a1fb729fa28ccee36cfa09ca7329dd591084d5a1b41f79573afded805858774b"}
Oct 30 21:44:16.558221 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:16.558181    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm" event={"ID":"b2c4b8e3-b6db-4489-9344-6b3fa5bdd778","Type":"ContainerStarted","Data":"f02892464ca40bda0f4af7fa363b4916a843e8d89a5b127ba027792b5d47548f"}
Oct 30 21:44:16.558631 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:16.558268    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm"
Oct 30 21:44:16.677699 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:16.677629    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm" podStartSLOduration=1.868877076 podStartE2EDuration="2.677608093s" totalImagesPullingTime="808.731017ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:16.676390191 +0000 UTC m=+599.348294125" watchObservedRunningTime="2026-10-30 21:44:16.677608093 +0000 UTC m=+599.349512030"
Oct 30 21:44:17.506890 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:17.506860    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 21:44:17.508371 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:17.508344    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 21:44:17.512638 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:17.512613    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 21:44:17.514304 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:17.514282    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 21:44:18.360398 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:44:18.360360    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:44:18.563312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:18.563259    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:21.525372 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:21.525342    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-l4jhz"
Oct 30 21:44:21.525875 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:21.525409    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-8m8pr"
Oct 30 21:44:26.366743 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.366702    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"]
Oct 30 21:44:26.370659 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.370631    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.434334 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.434300    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.434497 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.434348    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.434497 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.434369    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.434497 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.434464    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6930c8f5-9706-4db9-8957-8f209fc889a0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.434613 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.434511    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxnmk\" (UniqueName: \"kubernetes.io/projected/6930c8f5-9706-4db9-8957-8f209fc889a0-kube-api-access-kxnmk\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.434613 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.434537    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.434613 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.434570    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.434727 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.434666    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.434727 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.434699    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.470652 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.470608    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"]
Oct 30 21:44:26.535745 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.535713    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxnmk\" (UniqueName: \"kubernetes.io/projected/6930c8f5-9706-4db9-8957-8f209fc889a0-kube-api-access-kxnmk\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.536014 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.535755    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.536014 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.535781    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.536014 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.535812    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.536014 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.535851    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.536014 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.535874    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.536014 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.535921    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.536014 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.535944    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.536014 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.535984    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6930c8f5-9706-4db9-8957-8f209fc889a0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.536504 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.536179    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.536504 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.536359    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.536504 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.536390    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.536504 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.536457    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.536666 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.536649    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6930c8f5-9706-4db9-8957-8f209fc889a0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.538184 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.538161    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.538471 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.538454    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.594736 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.594697    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxnmk\" (UniqueName: \"kubernetes.io/projected/6930c8f5-9706-4db9-8957-8f209fc889a0-kube-api-access-kxnmk\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.595870 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.595850    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-mglq4\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:26.683431 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:26.683402    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:27.112478 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:27.112450    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"]
Oct 30 21:44:27.114246 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:44:27.114206    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6930c8f5_9706_4db9_8957_8f209fc889a0.slice/crio-d032f95078e9d8d9d8a965f10f2e158e9b78594e34c0ddce649de3346674d7fa WatchSource:0}: Error finding container d032f95078e9d8d9d8a965f10f2e158e9b78594e34c0ddce649de3346674d7fa: Status 404 returned error can't find the container with id d032f95078e9d8d9d8a965f10f2e158e9b78594e34c0ddce649de3346674d7fa
Oct 30 21:44:27.116412 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:27.116380    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:44:27.116485 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:27.116444    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:44:27.116485 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:27.116477    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:44:27.566568 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:27.566537    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-f74d5f7c9-mzjzm"
Oct 30 21:44:27.617633 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:27.617597    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4" event={"ID":"6930c8f5-9706-4db9-8957-8f209fc889a0","Type":"ContainerStarted","Data":"0043a83249f88e42fb2bc8892018dc434184f27ed438a98449e85604b405f668"}
Oct 30 21:44:27.617806 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:27.617656    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4" event={"ID":"6930c8f5-9706-4db9-8957-8f209fc889a0","Type":"ContainerStarted","Data":"d032f95078e9d8d9d8a965f10f2e158e9b78594e34c0ddce649de3346674d7fa"}
Oct 30 21:44:27.684539 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:27.684504    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:27.751610 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:27.751575    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-66dd76bf94-ktc28"]
Oct 30 21:44:27.753586 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:27.753548    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28" podUID="cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8" containerName="manager" containerID="cri-o://0eff629297a39d10920f5bc42966a5ec1328e262f4f98a284cc592ff0620b64a" gracePeriod=30
Oct 30 21:44:27.802014 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:27.801959    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4" podStartSLOduration=1.8019424960000001 podStartE2EDuration="1.801942496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:27.800617098 +0000 UTC m=+610.472521046" watchObservedRunningTime="2026-10-30 21:44:27.801942496 +0000 UTC m=+610.473846495"
Oct 30 21:44:28.066497 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.066472    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28"
Oct 30 21:44:28.153975 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.153947    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8-metrics-certs\") pod \"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8\" (UID: \"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8\") "
Oct 30 21:44:28.154123 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.154012    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8-kube-api-access-zcxzz\" (UniqueName: \"kubernetes.io/projected/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8-kube-api-access-zcxzz\") pod \"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8\" (UID: \"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8\") "
Oct 30 21:44:28.156262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.156203    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8-metrics-certs" pod "cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8" (UID: "cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:28.156386 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.156277    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8-kube-api-access-zcxzz" pod "cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8" (UID: "cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8"). InnerVolumeSpecName "kube-api-access-zcxzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:28.255171 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.255127    5586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8-metrics-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:28.255171 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.255164    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcxzz\" (UniqueName: \"kubernetes.io/projected/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8-kube-api-access-zcxzz\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:28.648508 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.648418    5586 generic.go:356] "Generic (PLEG): container finished" podID="cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8" containerID="0eff629297a39d10920f5bc42966a5ec1328e262f4f98a284cc592ff0620b64a" exitCode=0
Oct 30 21:44:28.648508 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.648477    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28"
Oct 30 21:44:28.648959 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.648513    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28" event={"ID":"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8","Type":"ContainerDied","Data":"0eff629297a39d10920f5bc42966a5ec1328e262f4f98a284cc592ff0620b64a"}
Oct 30 21:44:28.648959 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.648581    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-66dd76bf94-ktc28" event={"ID":"cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8","Type":"ContainerDied","Data":"d7e75daf7c6fddb6f1c7c2dbf55da801ce59f5bf0b6ad280d469987a66d6f4d4"}
Oct 30 21:44:28.648959 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.648617    5586 scope.go:122] "RemoveContainer" containerID="0eff629297a39d10920f5bc42966a5ec1328e262f4f98a284cc592ff0620b64a"
Oct 30 21:44:28.658702 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.658684    5586 scope.go:122] "RemoveContainer" containerID="e1a57751f771cea96de756e47bf6aba5fb5949cecb79ed0f234d05dc68d1343e"
Oct 30 21:44:28.667366 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.667342    5586 scope.go:122] "RemoveContainer" containerID="0eff629297a39d10920f5bc42966a5ec1328e262f4f98a284cc592ff0620b64a"
Oct 30 21:44:28.667656 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:44:28.667632    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eff629297a39d10920f5bc42966a5ec1328e262f4f98a284cc592ff0620b64a\": container with ID starting with 0eff629297a39d10920f5bc42966a5ec1328e262f4f98a284cc592ff0620b64a not found: ID does not exist" containerID="0eff629297a39d10920f5bc42966a5ec1328e262f4f98a284cc592ff0620b64a"
Oct 30 21:44:28.667702 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.667667    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eff629297a39d10920f5bc42966a5ec1328e262f4f98a284cc592ff0620b64a"} err="failed to get container status \"0eff629297a39d10920f5bc42966a5ec1328e262f4f98a284cc592ff0620b64a\": rpc error: code = NotFound desc = could not find container \"0eff629297a39d10920f5bc42966a5ec1328e262f4f98a284cc592ff0620b64a\": container with ID starting with 0eff629297a39d10920f5bc42966a5ec1328e262f4f98a284cc592ff0620b64a not found: ID does not exist"
Oct 30 21:44:28.667702 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.667687    5586 scope.go:122] "RemoveContainer" containerID="e1a57751f771cea96de756e47bf6aba5fb5949cecb79ed0f234d05dc68d1343e"
Oct 30 21:44:28.667946 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:44:28.667930    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1a57751f771cea96de756e47bf6aba5fb5949cecb79ed0f234d05dc68d1343e\": container with ID starting with e1a57751f771cea96de756e47bf6aba5fb5949cecb79ed0f234d05dc68d1343e not found: ID does not exist" containerID="e1a57751f771cea96de756e47bf6aba5fb5949cecb79ed0f234d05dc68d1343e"
Oct 30 21:44:28.667986 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.667950    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a57751f771cea96de756e47bf6aba5fb5949cecb79ed0f234d05dc68d1343e"} err="failed to get container status \"e1a57751f771cea96de756e47bf6aba5fb5949cecb79ed0f234d05dc68d1343e\": rpc error: code = NotFound desc = could not find container \"e1a57751f771cea96de756e47bf6aba5fb5949cecb79ed0f234d05dc68d1343e\": container with ID starting with e1a57751f771cea96de756e47bf6aba5fb5949cecb79ed0f234d05dc68d1343e not found: ID does not exist"
Oct 30 21:44:28.685884 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.685854    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 21:44:28.686008 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.685913    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4" podUID="6930c8f5-9706-4db9-8957-8f209fc889a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 21:44:28.812573 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.812539    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-66dd76bf94-ktc28"]
Oct 30 21:44:28.848600 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:28.848562    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-66dd76bf94-ktc28"]
Oct 30 21:44:29.451093 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:29.451060    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8" path="/var/lib/kubelet/pods/cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8/volumes"
Oct 30 21:44:29.546160 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:29.546110    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:29.684275 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:29.684219    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 21:44:29.684678 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:29.684295    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4" podUID="6930c8f5-9706-4db9-8957-8f209fc889a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 21:44:30.684355 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:30.684311    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 21:44:30.684766 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:30.684378    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4" podUID="6930c8f5-9706-4db9-8957-8f209fc889a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 21:44:31.684461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:31.684424    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 21:44:31.684849 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:31.684488    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4" podUID="6930c8f5-9706-4db9-8957-8f209fc889a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 21:44:32.684427 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:32.684386    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 21:44:32.684622 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:32.684446    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4" podUID="6930c8f5-9706-4db9-8957-8f209fc889a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 21:44:33.684777 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:33.684738    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 21:44:33.685306 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:33.684802    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4" podUID="6930c8f5-9706-4db9-8957-8f209fc889a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 21:44:34.684765 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:34.684728    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-mglq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 21:44:34.684975 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:34.684787    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4" podUID="6930c8f5-9706-4db9-8957-8f209fc889a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 21:44:35.688794 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:35.688762    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:35.689212 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:35.689040    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:35.689720 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:35.689702    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:44:40.081447 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:40.081410    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:48.359699 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:44:48.359660    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:44:51.675912 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:51.675866    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:51.904056 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:51.904016    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:53.401821 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:53.401783    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:53.581916 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:44:53.581874    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:00.179664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.179583    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"]
Oct 30 21:45:00.180106 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.180033    5586 cpu_manager.go:414] "containerMap: removing container" podUID="cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8" containerName="manager"
Oct 30 21:45:00.180106 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.180045    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8" containerName="manager"
Oct 30 21:45:00.180176 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.180121    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8" containerName="manager"
Oct 30 21:45:00.180176 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.180131    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8" containerName="manager"
Oct 30 21:45:00.186312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.186291    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"
Oct 30 21:45:00.244760 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.244724    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"]
Oct 30 21:45:00.291568 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.291540    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:45:00.291568 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.291551    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:45:00.291788 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.291582    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 21:45:00.361663 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.361624    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkfcp\" (UniqueName: \"kubernetes.io/projected/31a957a1-4647-4586-a1ef-072754cc8191-kube-api-access-hkfcp\") pod \"maas-api-key-cleanup-29889945-xrfd7\" (UID: \"31a957a1-4647-4586-a1ef-072754cc8191\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"
Oct 30 21:45:00.361843 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.361789    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31a957a1-4647-4586-a1ef-072754cc8191-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889945-xrfd7\" (UID: \"31a957a1-4647-4586-a1ef-072754cc8191\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"
Oct 30 21:45:00.462474 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.462390    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkfcp\" (UniqueName: \"kubernetes.io/projected/31a957a1-4647-4586-a1ef-072754cc8191-kube-api-access-hkfcp\") pod \"maas-api-key-cleanup-29889945-xrfd7\" (UID: \"31a957a1-4647-4586-a1ef-072754cc8191\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"
Oct 30 21:45:00.462630 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.462488    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31a957a1-4647-4586-a1ef-072754cc8191-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889945-xrfd7\" (UID: \"31a957a1-4647-4586-a1ef-072754cc8191\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"
Oct 30 21:45:00.463055 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.463035    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31a957a1-4647-4586-a1ef-072754cc8191-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889945-xrfd7\" (UID: \"31a957a1-4647-4586-a1ef-072754cc8191\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"
Oct 30 21:45:00.520057 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.520025    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkfcp\" (UniqueName: \"kubernetes.io/projected/31a957a1-4647-4586-a1ef-072754cc8191-kube-api-access-hkfcp\") pod \"maas-api-key-cleanup-29889945-xrfd7\" (UID: \"31a957a1-4647-4586-a1ef-072754cc8191\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"
Oct 30 21:45:00.799525 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:00.799429    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"
Oct 30 21:45:01.172298 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:01.172268    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"]
Oct 30 21:45:01.174914 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:45:01.174887    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31a957a1_4647_4586_a1ef_072754cc8191.slice/crio-5fba5e59b21633bb4a45c8ff0998fe9191ba2786717e230bf94e920563173d64 WatchSource:0}: Error finding container 5fba5e59b21633bb4a45c8ff0998fe9191ba2786717e230bf94e920563173d64: Status 404 returned error can't find the container with id 5fba5e59b21633bb4a45c8ff0998fe9191ba2786717e230bf94e920563173d64
Oct 30 21:45:01.834422 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:01.834388    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7" event={"ID":"31a957a1-4647-4586-a1ef-072754cc8191","Type":"ContainerStarted","Data":"5fba5e59b21633bb4a45c8ff0998fe9191ba2786717e230bf94e920563173d64"}
Oct 30 21:45:05.859767 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:05.859728    5586 generic.go:356] "Generic (PLEG): container finished" podID="31a957a1-4647-4586-a1ef-072754cc8191" containerID="318bab6c5915c317d2a5f7edffeadf1eec59fa641e1a612d4a9ce44499aecb13" exitCode=60
Oct 30 21:45:05.860171 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:05.859814    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7" event={"ID":"31a957a1-4647-4586-a1ef-072754cc8191","Type":"ContainerDied","Data":"318bab6c5915c317d2a5f7edffeadf1eec59fa641e1a612d4a9ce44499aecb13"}
Oct 30 21:45:05.860171 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:05.860029    5586 scope.go:122] "RemoveContainer" containerID="318bab6c5915c317d2a5f7edffeadf1eec59fa641e1a612d4a9ce44499aecb13"
Oct 30 21:45:06.868780 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:06.868742    5586 generic.go:356] "Generic (PLEG): container finished" podID="31a957a1-4647-4586-a1ef-072754cc8191" containerID="24cebbdefa1f5f200f497e6a6ff4e2e78747edfe609cb7d4fdba7eda1cc8b19e" exitCode=60
Oct 30 21:45:06.869209 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:06.868800    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7" event={"ID":"31a957a1-4647-4586-a1ef-072754cc8191","Type":"ContainerDied","Data":"24cebbdefa1f5f200f497e6a6ff4e2e78747edfe609cb7d4fdba7eda1cc8b19e"}
Oct 30 21:45:06.869209 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:06.868863    5586 scope.go:122] "RemoveContainer" containerID="318bab6c5915c317d2a5f7edffeadf1eec59fa641e1a612d4a9ce44499aecb13"
Oct 30 21:45:06.869209 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:06.869107    5586 scope.go:122] "RemoveContainer" containerID="24cebbdefa1f5f200f497e6a6ff4e2e78747edfe609cb7d4fdba7eda1cc8b19e"
Oct 30 21:45:06.869418 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:45:06.869393    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889945-xrfd7_odh-ai-gateway-infra(31a957a1-4647-4586-a1ef-072754cc8191)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7" podUID="31a957a1-4647-4586-a1ef-072754cc8191"
Oct 30 21:45:07.875904 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:07.875873    5586 scope.go:122] "RemoveContainer" containerID="24cebbdefa1f5f200f497e6a6ff4e2e78747edfe609cb7d4fdba7eda1cc8b19e"
Oct 30 21:45:07.876315 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:45:07.876121    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889945-xrfd7_odh-ai-gateway-infra(31a957a1-4647-4586-a1ef-072754cc8191)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7" podUID="31a957a1-4647-4586-a1ef-072754cc8191"
Oct 30 21:45:18.361709 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:45:18.361664    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:45:18.440429 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:18.440398    5586 scope.go:122] "RemoveContainer" containerID="24cebbdefa1f5f200f497e6a6ff4e2e78747edfe609cb7d4fdba7eda1cc8b19e"
Oct 30 21:45:18.902536 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:18.902450    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:18.944503 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:18.944467    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7" event={"ID":"31a957a1-4647-4586-a1ef-072754cc8191","Type":"ContainerStarted","Data":"706eecb3a410f385fa0ab4ff211b0e54d0dbf36cf4604fd600d1ba56dc4e5360"}
Oct 30 21:45:19.060706 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:19.060652    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7" podStartSLOduration=15.263689656 podStartE2EDuration="19.060637748s" totalImagesPullingTime="3.796948092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:19.05961929 +0000 UTC m=+661.731523226" watchObservedRunningTime="2026-10-30 21:45:19.060637748 +0000 UTC m=+661.732541683"
Oct 30 21:45:19.952291 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:19.952257    5586 generic.go:356] "Generic (PLEG): container finished" podID="31a957a1-4647-4586-a1ef-072754cc8191" containerID="706eecb3a410f385fa0ab4ff211b0e54d0dbf36cf4604fd600d1ba56dc4e5360" exitCode=60
Oct 30 21:45:19.952738 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:19.952262    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7" event={"ID":"31a957a1-4647-4586-a1ef-072754cc8191","Type":"ContainerDied","Data":"706eecb3a410f385fa0ab4ff211b0e54d0dbf36cf4604fd600d1ba56dc4e5360"}
Oct 30 21:45:19.952738 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:19.952399    5586 scope.go:122] "RemoveContainer" containerID="24cebbdefa1f5f200f497e6a6ff4e2e78747edfe609cb7d4fdba7eda1cc8b19e"
Oct 30 21:45:19.952738 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:19.952732    5586 scope.go:122] "RemoveContainer" containerID="706eecb3a410f385fa0ab4ff211b0e54d0dbf36cf4604fd600d1ba56dc4e5360"
Oct 30 21:45:19.953053 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:45:19.953030    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889945-xrfd7_odh-ai-gateway-infra(31a957a1-4647-4586-a1ef-072754cc8191)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7" podUID="31a957a1-4647-4586-a1ef-072754cc8191"
Oct 30 21:45:20.077670 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:20.077633    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"]
Oct 30 21:45:21.109458 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:21.109431    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"
Oct 30 21:45:21.175532 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:21.175493    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31a957a1-4647-4586-a1ef-072754cc8191-kube-api-access-hkfcp\" (UniqueName: \"kubernetes.io/projected/31a957a1-4647-4586-a1ef-072754cc8191-kube-api-access-hkfcp\") pod \"31a957a1-4647-4586-a1ef-072754cc8191\" (UID: \"31a957a1-4647-4586-a1ef-072754cc8191\") "
Oct 30 21:45:21.175732 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:21.175608    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31a957a1-4647-4586-a1ef-072754cc8191-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31a957a1-4647-4586-a1ef-072754cc8191-maas-api-service-ca\") pod \"31a957a1-4647-4586-a1ef-072754cc8191\" (UID: \"31a957a1-4647-4586-a1ef-072754cc8191\") "
Oct 30 21:45:21.175964 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:21.175941    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31a957a1-4647-4586-a1ef-072754cc8191-maas-api-service-ca" pod "31a957a1-4647-4586-a1ef-072754cc8191" (UID: "31a957a1-4647-4586-a1ef-072754cc8191"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:21.177744 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:21.177722    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31a957a1-4647-4586-a1ef-072754cc8191-kube-api-access-hkfcp" pod "31a957a1-4647-4586-a1ef-072754cc8191" (UID: "31a957a1-4647-4586-a1ef-072754cc8191"). InnerVolumeSpecName "kube-api-access-hkfcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:21.276734 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:21.276650    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkfcp\" (UniqueName: \"kubernetes.io/projected/31a957a1-4647-4586-a1ef-072754cc8191-kube-api-access-hkfcp\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:21.276734 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:21.276681    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31a957a1-4647-4586-a1ef-072754cc8191-maas-api-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:21.767163 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:21.767102    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal" podUID="6ea3e08f52bca920602c76ad35bcaf69" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Oct 30 21:45:21.966177 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:21.966148    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"
Oct 30 21:45:21.966353 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:21.966189    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7" event={"ID":"31a957a1-4647-4586-a1ef-072754cc8191","Type":"ContainerDied","Data":"5fba5e59b21633bb4a45c8ff0998fe9191ba2786717e230bf94e920563173d64"}
Oct 30 21:45:21.966353 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:21.966303    5586 scope.go:122] "RemoveContainer" containerID="706eecb3a410f385fa0ab4ff211b0e54d0dbf36cf4604fd600d1ba56dc4e5360"
Oct 30 21:45:22.128259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:22.128156    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"]
Oct 30 21:45:22.177339 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:22.177295    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889945-xrfd7"]
Oct 30 21:45:23.446225 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:23.446191    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31a957a1-4647-4586-a1ef-072754cc8191" path="/var/lib/kubelet/pods/31a957a1-4647-4586-a1ef-072754cc8191/volumes"
Oct 30 21:45:38.617860 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:38.617805    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:38.787627 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:38.787592    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:39.183196 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.183165    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"]
Oct 30 21:45:39.183599 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.183586    5586 cpu_manager.go:414] "containerMap: removing container" podUID="31a957a1-4647-4586-a1ef-072754cc8191" containerName="cleanup"
Oct 30 21:45:39.183644 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.183600    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31a957a1-4647-4586-a1ef-072754cc8191" containerName="cleanup"
Oct 30 21:45:39.183644 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.183634    5586 cpu_manager.go:414] "containerMap: removing container" podUID="cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8" containerName="manager"
Oct 30 21:45:39.183644 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.183639    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd12bf7f-0fc3-43b6-a8fc-b68aafb37fb8" containerName="manager"
Oct 30 21:45:39.183734 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.183645    5586 cpu_manager.go:414] "containerMap: removing container" podUID="31a957a1-4647-4586-a1ef-072754cc8191" containerName="cleanup"
Oct 30 21:45:39.183734 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.183650    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31a957a1-4647-4586-a1ef-072754cc8191" containerName="cleanup"
Oct 30 21:45:39.183734 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.183713    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="31a957a1-4647-4586-a1ef-072754cc8191" containerName="cleanup"
Oct 30 21:45:39.183734 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.183719    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="31a957a1-4647-4586-a1ef-072754cc8191" containerName="cleanup"
Oct 30 21:45:39.191322 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.191302    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.244327 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.244270    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-557708-openshift-default-dockercfg-87jdg\""
Oct 30 21:45:39.301496 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.301461    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"]
Oct 30 21:45:39.312421 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.312378    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:39.352698 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.352655    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lqkf\" (UniqueName: \"kubernetes.io/projected/37583645-5073-494b-8079-7eac9ceb5d30-kube-api-access-6lqkf\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.352875 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.352707    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37583645-5073-494b-8079-7eac9ceb5d30-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.352875 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.352802    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.352947 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.352892    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37583645-5073-494b-8079-7eac9ceb5d30-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.352947 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.352935    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.353016 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.352970    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.353016 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.353011    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.353077 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.353046    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.353112 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.353081    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37583645-5073-494b-8079-7eac9ceb5d30-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.454091 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.454000    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.454091 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.454038    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.454091 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.454061    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.454435 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.454096    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37583645-5073-494b-8079-7eac9ceb5d30-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.454435 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.454127    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lqkf\" (UniqueName: \"kubernetes.io/projected/37583645-5073-494b-8079-7eac9ceb5d30-kube-api-access-6lqkf\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.454435 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.454163    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37583645-5073-494b-8079-7eac9ceb5d30-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.454435 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.454204    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.454435 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.454293    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37583645-5073-494b-8079-7eac9ceb5d30-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.454435 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.454348    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.454711 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.454474    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.454711 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.454502    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.454711 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.454552    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.454711 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.454574    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.455047 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.455021    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37583645-5073-494b-8079-7eac9ceb5d30-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.456654 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.456631    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.456952 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.456931    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37583645-5073-494b-8079-7eac9ceb5d30-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.512206 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.512170    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"]
Oct 30 21:45:39.512797 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.512763    5586 cpu_manager.go:414] "containerMap: removing container" podUID="31a957a1-4647-4586-a1ef-072754cc8191" containerName="cleanup"
Oct 30 21:45:39.512797 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.512785    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31a957a1-4647-4586-a1ef-072754cc8191" containerName="cleanup"
Oct 30 21:45:39.512975 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.512861    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="31a957a1-4647-4586-a1ef-072754cc8191" containerName="cleanup"
Oct 30 21:45:39.516388 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.516358    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37583645-5073-494b-8079-7eac9ceb5d30-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.516526 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.516460    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.519126 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.519096    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lqkf\" (UniqueName: \"kubernetes.io/projected/37583645-5073-494b-8079-7eac9ceb5d30-kube-api-access-6lqkf\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.576803 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.576770    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"]
Oct 30 21:45:39.578488 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.578460    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-c3e2d8-openshift-default-dockercfg-hqs2q\""
Oct 30 21:45:39.626685 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.626639    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:39.656838 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.656801    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.657031 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.656854    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.657031 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.656925    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.657031 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.657006    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.657133 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.657046    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s47tj\" (UniqueName: \"kubernetes.io/projected/cd9299f1-9812-4f6c-8851-6957b8b3ab64-kube-api-access-s47tj\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.657133 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.657081    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.657211 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.657161    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.657211 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.657203    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.657312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.657219    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.758843 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.758741    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.758843 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.758798    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.759050 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.758906    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.759050 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.759004    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.759050 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.759030    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.759193 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.759062    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.759193 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.759100    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.759193 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.759131    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s47tj\" (UniqueName: \"kubernetes.io/projected/cd9299f1-9812-4f6c-8851-6957b8b3ab64-kube-api-access-s47tj\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.759193 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.759177    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.759439 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.759198    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.759439 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.759228    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.759547 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.759477    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.759547 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.759533    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.759818 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.759799    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.761250 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.761220    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.761412 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.761394    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.805473 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.805433    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:39.810525 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.810486    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:39.816533 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.816513    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.819791 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.819772    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s47tj\" (UniqueName: \"kubernetes.io/projected/cd9299f1-9812-4f6c-8851-6957b8b3ab64-kube-api-access-s47tj\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.849903 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.849861    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:39.977534 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:39.977496    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:40.178100 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.178013    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:40.242416 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.242385    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"]
Oct 30 21:45:40.280911 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:45:40.280882    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37583645_5073_494b_8079_7eac9ceb5d30.slice/crio-4de8192824b9f9d993604332f60b31963ce7b29ae953d6eda407b6a963d9646e WatchSource:0}: Error finding container 4de8192824b9f9d993604332f60b31963ce7b29ae953d6eda407b6a963d9646e: Status 404 returned error can't find the container with id 4de8192824b9f9d993604332f60b31963ce7b29ae953d6eda407b6a963d9646e
Oct 30 21:45:40.283263 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.283201    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:40.283361 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.283296    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:40.283361 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.283326    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:40.295533 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.295413    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"]
Oct 30 21:45:40.296842 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:45:40.296802    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd9299f1_9812_4f6c_8851_6957b8b3ab64.slice/crio-7bcf250a2a0b73cc6db0eee96e257346f23e6c246ef17fbd03028fcb9c8785c3 WatchSource:0}: Error finding container 7bcf250a2a0b73cc6db0eee96e257346f23e6c246ef17fbd03028fcb9c8785c3: Status 404 returned error can't find the container with id 7bcf250a2a0b73cc6db0eee96e257346f23e6c246ef17fbd03028fcb9c8785c3
Oct 30 21:45:40.301113 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.300612    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:40.301113 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.300701    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:40.301113 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.300744    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:40.331378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.331351    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:40.467563 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.467467    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"]
Oct 30 21:45:40.471960 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.471932    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.524959 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.524927    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-9b51e1-openshift-default-dockercfg-b9z85\""
Oct 30 21:45:40.539501 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.539463    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"]
Oct 30 21:45:40.568140 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.568096    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.568439 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.568409    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmkkx\" (UniqueName: \"kubernetes.io/projected/93475d2f-6f42-47e4-9c1e-44cd9080924c-kube-api-access-hmkkx\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.568552 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.568465    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.568609 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.568588    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.568663 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.568648    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.568772 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.568751    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.568831 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.568806    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93475d2f-6f42-47e4-9c1e-44cd9080924c-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.568953 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.568936    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.568993 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.568966    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.572354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.572324    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:40.669671 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.669640    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.669671 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.669678    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.670294 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.669802    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.670294 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.669844    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmkkx\" (UniqueName: \"kubernetes.io/projected/93475d2f-6f42-47e4-9c1e-44cd9080924c-kube-api-access-hmkkx\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.670294 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.669876    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.670294 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.669911    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.670294 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.669941    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.670294 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.669970    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.670294 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.670016    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93475d2f-6f42-47e4-9c1e-44cd9080924c-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.670294 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.670030    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.670607 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.670299    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.670607 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.670349    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.670607 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.670567    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.670754 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.670736    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93475d2f-6f42-47e4-9c1e-44cd9080924c-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.672284 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.672264    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.672516 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.672499    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.727508 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.727439    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.727655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.727635    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmkkx\" (UniqueName: \"kubernetes.io/projected/93475d2f-6f42-47e4-9c1e-44cd9080924c-kube-api-access-hmkkx\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.731224 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.731187    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:40.783510 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.783474    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"]
Oct 30 21:45:40.788480 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.788459    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:40.788686 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.788666    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:40.848356 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.848316    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"]
Oct 30 21:45:40.883736 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.883700    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-bc009f-openshift-default-dockercfg-4jwnk\""
Oct 30 21:45:40.895735 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.895702    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:40.973336 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.973299    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26xxh\" (UniqueName: \"kubernetes.io/projected/b40a8985-4520-4dbc-894d-b07e94d9deac-kube-api-access-26xxh\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:40.973521 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.973359    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:40.973521 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.973487    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:40.973619 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.973522    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:40.973619 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.973543    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:40.973773 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.973621    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:40.973773 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.973660    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:40.973773 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.973743    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:40.973773 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:40.973760    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b40a8985-4520-4dbc-894d-b07e94d9deac-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.045756 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.045671    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:41.075484 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.075448    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26xxh\" (UniqueName: \"kubernetes.io/projected/b40a8985-4520-4dbc-894d-b07e94d9deac-kube-api-access-26xxh\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.075670 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.075527    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.075670 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.075583    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.075670 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.075615    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.075670 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.075647    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.075881 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.075686    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.075881 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.075748    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.075881 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.075792    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.075881 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.075817    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b40a8985-4520-4dbc-894d-b07e94d9deac-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.076101 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.075995    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.076101 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.076031    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.076101 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.076073    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.076346 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.076326    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.076575 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.076553    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b40a8985-4520-4dbc-894d-b07e94d9deac-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.077965 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.077947    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.078394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.078372    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.081203 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.081176    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf" event={"ID":"37583645-5073-494b-8079-7eac9ceb5d30","Type":"ContainerStarted","Data":"1e5933086ae37e06e5887a52e8c3336a2e5e5992bbca81f0c5c22f870da44bb0"}
Oct 30 21:45:41.081331 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.081249    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf" event={"ID":"37583645-5073-494b-8079-7eac9ceb5d30","Type":"ContainerStarted","Data":"4de8192824b9f9d993604332f60b31963ce7b29ae953d6eda407b6a963d9646e"}
Oct 30 21:45:41.082816 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.082794    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd" event={"ID":"cd9299f1-9812-4f6c-8851-6957b8b3ab64","Type":"ContainerStarted","Data":"803b026b4bc0d8aa4d34faa0bef1c98e42c98b35add07399fd76f20e4bf51ab3"}
Oct 30 21:45:41.082897 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.082839    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd" event={"ID":"cd9299f1-9812-4f6c-8851-6957b8b3ab64","Type":"ContainerStarted","Data":"7bcf250a2a0b73cc6db0eee96e257346f23e6c246ef17fbd03028fcb9c8785c3"}
Oct 30 21:45:41.136208 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.136174    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.136380 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.136265    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26xxh\" (UniqueName: \"kubernetes.io/projected/b40a8985-4520-4dbc-894d-b07e94d9deac-kube-api-access-26xxh\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.179591 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.179563    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"]
Oct 30 21:45:41.180767 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:45:41.180744    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93475d2f_6f42_47e4_9c1e_44cd9080924c.slice/crio-027450e283f474e10c36d8e4716d7d8ab02b54719c45d95534d79fc2840d35a8 WatchSource:0}: Error finding container 027450e283f474e10c36d8e4716d7d8ab02b54719c45d95534d79fc2840d35a8: Status 404 returned error can't find the container with id 027450e283f474e10c36d8e4716d7d8ab02b54719c45d95534d79fc2840d35a8
Oct 30 21:45:41.183114 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.183084    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:41.183196 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.183146    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:41.183196 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.183176    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:41.218925 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.218873    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf" podStartSLOduration=2.218857289 podStartE2EDuration="2.218857289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:41.216458881 +0000 UTC m=+683.888362815" watchObservedRunningTime="2026-10-30 21:45:41.218857289 +0000 UTC m=+683.890761226"
Oct 30 21:45:41.254216 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.254180    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:41.359825 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.359687    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd" podStartSLOduration=2.359665648 podStartE2EDuration="2.359665648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:41.350619324 +0000 UTC m=+684.022523257" watchObservedRunningTime="2026-10-30 21:45:41.359665648 +0000 UTC m=+684.031569587"
Oct 30 21:45:41.404434 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.404395    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:41.430591 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.430554    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:41.786126 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.786080    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"]
Oct 30 21:45:41.789575 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:45:41.789546    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40a8985_4520_4dbc_894d_b07e94d9deac.slice/crio-28dd7b9caa8ba0a4015e92be34232b58cc1bc65be68e8c88c93036d0720b8dd0 WatchSource:0}: Error finding container 28dd7b9caa8ba0a4015e92be34232b58cc1bc65be68e8c88c93036d0720b8dd0: Status 404 returned error can't find the container with id 28dd7b9caa8ba0a4015e92be34232b58cc1bc65be68e8c88c93036d0720b8dd0
Oct 30 21:45:41.792330 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.792298    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:41.792403 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.792368    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:41.792443 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.792401    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:45:41.806370 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.806347    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:41.812290 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.812271    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:41.850168 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.850148    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:41.855575 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:41.855552    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:42.091859 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:42.091758    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" event={"ID":"b40a8985-4520-4dbc-894d-b07e94d9deac","Type":"ContainerStarted","Data":"67efb1ff70be31453fb0b03d24f694c7cf42d57cd50d0effc4a938b0eaf29e08"}
Oct 30 21:45:42.091859 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:42.091822    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" event={"ID":"b40a8985-4520-4dbc-894d-b07e94d9deac","Type":"ContainerStarted","Data":"28dd7b9caa8ba0a4015e92be34232b58cc1bc65be68e8c88c93036d0720b8dd0"}
Oct 30 21:45:42.093600 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:42.093565    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5" event={"ID":"93475d2f-6f42-47e4-9c1e-44cd9080924c","Type":"ContainerStarted","Data":"13a09357e9e29b57a29134c2c276d66a096fddf2e9d2aa3d197a7cd71499f430"}
Oct 30 21:45:42.094013 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:42.093985    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:42.094161 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:42.094036    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5" event={"ID":"93475d2f-6f42-47e4-9c1e-44cd9080924c","Type":"ContainerStarted","Data":"027450e283f474e10c36d8e4716d7d8ab02b54719c45d95534d79fc2840d35a8"}
Oct 30 21:45:42.094161 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:42.094067    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:42.094849 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:42.094822    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:45:42.094966 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:42.094924    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:45:42.215585 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:42.215508    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" podStartSLOduration=2.215488735 podStartE2EDuration="2.215488735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:42.214725476 +0000 UTC m=+684.886629412" watchObservedRunningTime="2026-10-30 21:45:42.215488735 +0000 UTC m=+684.887392673"
Oct 30 21:45:42.336315 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:42.335476    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5" podStartSLOduration=2.335454909 podStartE2EDuration="2.335454909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:42.334160696 +0000 UTC m=+685.006064633" watchObservedRunningTime="2026-10-30 21:45:42.335454909 +0000 UTC m=+685.007358846"
Oct 30 21:45:42.405341 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:42.405313    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:42.410811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:42.410785    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:42.789825 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:42.789738    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:42.794979 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:42.794951    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:43.103527 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:43.103335    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:43.103527 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:43.103431    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:43.104073 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:43.104050    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:45:43.104331 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:43.104310    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:45:43.982650 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:43.982614    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:44.157373 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:44.157343    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:45.000554 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:45.000515    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:45.214081 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:45.214024    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:45.408829 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:45.408785    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:45.815717 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:45.815637    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:46.061216 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:46.061184    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:47.070517 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:47.070479    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:47.435214 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:47.435181    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:47.667871 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:47.667834    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:47.898445 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:47.898311    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:48.091078 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.091042    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:48.268876 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.268838    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:48.362732 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:45:48.362693    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:45:48.424966 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.424927    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"]
Oct 30 21:45:48.430351 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.430318    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"
Oct 30 21:45:48.456552 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.456520    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4ab435-292c-4532-a4de-412d74acc5ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx\" (UID: \"2d4ab435-292c-4532-a4de-412d74acc5ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"
Oct 30 21:45:48.456770 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.456745    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqxgr\" (UniqueName: \"kubernetes.io/projected/2d4ab435-292c-4532-a4de-412d74acc5ae-kube-api-access-kqxgr\") pod \"payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx\" (UID: \"2d4ab435-292c-4532-a4de-412d74acc5ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"
Oct 30 21:45:48.460382 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.460346    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:48.486571 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.486532    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"]
Oct 30 21:45:48.532656 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.532546    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-c79cb5-dockercfg-28htn\""
Oct 30 21:45:48.533024 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.532986    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-c79cb5\""
Oct 30 21:45:48.558111 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.558066    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqxgr\" (UniqueName: \"kubernetes.io/projected/2d4ab435-292c-4532-a4de-412d74acc5ae-kube-api-access-kqxgr\") pod \"payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx\" (UID: \"2d4ab435-292c-4532-a4de-412d74acc5ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"
Oct 30 21:45:48.558340 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.558136    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4ab435-292c-4532-a4de-412d74acc5ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx\" (UID: \"2d4ab435-292c-4532-a4de-412d74acc5ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"
Oct 30 21:45:48.558947 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.558919    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4ab435-292c-4532-a4de-412d74acc5ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx\" (UID: \"2d4ab435-292c-4532-a4de-412d74acc5ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"
Oct 30 21:45:48.584260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.584214    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"]
Oct 30 21:45:48.588199 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.588173    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"
Oct 30 21:45:48.614768 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.614740    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqxgr\" (UniqueName: \"kubernetes.io/projected/2d4ab435-292c-4532-a4de-412d74acc5ae-kube-api-access-kqxgr\") pod \"payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx\" (UID: \"2d4ab435-292c-4532-a4de-412d74acc5ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"
Oct 30 21:45:48.646266 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.646218    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"]
Oct 30 21:45:48.659398 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.659367    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0622aae-1702-4cb1-80b6-8740c5261667-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh\" (UID: \"a0622aae-1702-4cb1-80b6-8740c5261667\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"
Oct 30 21:45:48.659398 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.659400    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zggn\" (UniqueName: \"kubernetes.io/projected/a0622aae-1702-4cb1-80b6-8740c5261667-kube-api-access-7zggn\") pod \"payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh\" (UID: \"a0622aae-1702-4cb1-80b6-8740c5261667\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"
Oct 30 21:45:48.745456 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.745418    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"
Oct 30 21:45:48.761069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.761031    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0622aae-1702-4cb1-80b6-8740c5261667-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh\" (UID: \"a0622aae-1702-4cb1-80b6-8740c5261667\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"
Oct 30 21:45:48.761206 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.761080    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zggn\" (UniqueName: \"kubernetes.io/projected/a0622aae-1702-4cb1-80b6-8740c5261667-kube-api-access-7zggn\") pod \"payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh\" (UID: \"a0622aae-1702-4cb1-80b6-8740c5261667\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"
Oct 30 21:45:48.761747 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.761722    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0622aae-1702-4cb1-80b6-8740c5261667-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh\" (UID: \"a0622aae-1702-4cb1-80b6-8740c5261667\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"
Oct 30 21:45:48.819010 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.818983    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zggn\" (UniqueName: \"kubernetes.io/projected/a0622aae-1702-4cb1-80b6-8740c5261667-kube-api-access-7zggn\") pod \"payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh\" (UID: \"a0622aae-1702-4cb1-80b6-8740c5261667\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"
Oct 30 21:45:48.901681 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:48.901632    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"
Oct 30 21:45:49.136988 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:49.136960    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"]
Oct 30 21:45:49.138125 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:45:49.138096    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d4ab435_292c_4532_a4de_412d74acc5ae.slice/crio-386680365c9741a344b3756016997035aea45519130663fc71f1d95950e6e341 WatchSource:0}: Error finding container 386680365c9741a344b3756016997035aea45519130663fc71f1d95950e6e341: Status 404 returned error can't find the container with id 386680365c9741a344b3756016997035aea45519130663fc71f1d95950e6e341
Oct 30 21:45:49.283835 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:49.283769    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"]
Oct 30 21:45:49.286269 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:45:49.286219    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0622aae_1702_4cb1_80b6_8740c5261667.slice/crio-46c9c22e53aee14281cf3b03c7e45f949a6a613008374262843215a806032205 WatchSource:0}: Error finding container 46c9c22e53aee14281cf3b03c7e45f949a6a613008374262843215a806032205: Status 404 returned error can't find the container with id 46c9c22e53aee14281cf3b03c7e45f949a6a613008374262843215a806032205
Oct 30 21:45:49.938854 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:49.938820    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:50.150032 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:50.149987    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh" event={"ID":"a0622aae-1702-4cb1-80b6-8740c5261667","Type":"ContainerStarted","Data":"18b92ea91d1fdaf4539ad1e91021dde2b92320eb37de3472069efa1eded9f3dc"}
Oct 30 21:45:50.150489 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:50.150040    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh" event={"ID":"a0622aae-1702-4cb1-80b6-8740c5261667","Type":"ContainerStarted","Data":"46c9c22e53aee14281cf3b03c7e45f949a6a613008374262843215a806032205"}
Oct 30 21:45:50.150489 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:50.150156    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"
Oct 30 21:45:50.151584 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:50.151561    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx" event={"ID":"2d4ab435-292c-4532-a4de-412d74acc5ae","Type":"ContainerStarted","Data":"9c6fdba8a0f37b44706ee7aad0f8d908c8cfc31c2543411f0071c952c3918a9f"}
Oct 30 21:45:50.151699 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:50.151608    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx" event={"ID":"2d4ab435-292c-4532-a4de-412d74acc5ae","Type":"ContainerStarted","Data":"386680365c9741a344b3756016997035aea45519130663fc71f1d95950e6e341"}
Oct 30 21:45:50.151699 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:50.151655    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"
Oct 30 21:45:50.168652 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:50.168618    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:50.267677 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:50.267574    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh" podStartSLOduration=2.267558007 podStartE2EDuration="2.267558007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:50.265696692 +0000 UTC m=+692.937600626" watchObservedRunningTime="2026-10-30 21:45:50.267558007 +0000 UTC m=+692.939461942"
Oct 30 21:45:50.622124 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:50.622040    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:50.801034 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:50.801002    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:50.967417 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:50.967383    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:51.055673 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.055618    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx" podStartSLOduration=3.055602649 podStartE2EDuration="3.055602649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:50.391402104 +0000 UTC m=+693.063306039" watchObservedRunningTime="2026-10-30 21:45:51.055602649 +0000 UTC m=+693.727506585"
Oct 30 21:45:51.057412 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.057389    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"]
Oct 30 21:45:51.061096 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.061078    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"
Oct 30 21:45:51.093382 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.093342    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33ecbfc9-fecd-4f7a-b162-40bd00a040b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h\" (UID: \"33ecbfc9-fecd-4f7a-b162-40bd00a040b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"
Oct 30 21:45:51.093561 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.093410    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7gb9\" (UniqueName: \"kubernetes.io/projected/33ecbfc9-fecd-4f7a-b162-40bd00a040b9-kube-api-access-m7gb9\") pod \"payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h\" (UID: \"33ecbfc9-fecd-4f7a-b162-40bd00a040b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"
Oct 30 21:45:51.109325 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.109290    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"]
Oct 30 21:45:51.113960 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.113932    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"
Oct 30 21:45:51.116902 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.116872    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-557708\""
Oct 30 21:45:51.117076 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.117023    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-557708-dockercfg-zq59s\""
Oct 30 21:45:51.164752 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.163692    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"]
Oct 30 21:45:51.173693 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.173660    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"]
Oct 30 21:45:51.194887 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.194852    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33ecbfc9-fecd-4f7a-b162-40bd00a040b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h\" (UID: \"33ecbfc9-fecd-4f7a-b162-40bd00a040b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"
Oct 30 21:45:51.194887 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.194890    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlrm8\" (UniqueName: \"kubernetes.io/projected/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541-kube-api-access-xlrm8\") pod \"payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp\" (UID: \"9c7d9808-fd6c-480c-8b6a-1f3a47eb6541\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"
Oct 30 21:45:51.195114 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.195000    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp\" (UID: \"9c7d9808-fd6c-480c-8b6a-1f3a47eb6541\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"
Oct 30 21:45:51.195114 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.195109    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7gb9\" (UniqueName: \"kubernetes.io/projected/33ecbfc9-fecd-4f7a-b162-40bd00a040b9-kube-api-access-m7gb9\") pod \"payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h\" (UID: \"33ecbfc9-fecd-4f7a-b162-40bd00a040b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"
Oct 30 21:45:51.195727 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.195704    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33ecbfc9-fecd-4f7a-b162-40bd00a040b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h\" (UID: \"33ecbfc9-fecd-4f7a-b162-40bd00a040b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"
Oct 30 21:45:51.252104 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.252026    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7gb9\" (UniqueName: \"kubernetes.io/projected/33ecbfc9-fecd-4f7a-b162-40bd00a040b9-kube-api-access-m7gb9\") pod \"payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h\" (UID: \"33ecbfc9-fecd-4f7a-b162-40bd00a040b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"
Oct 30 21:45:51.295990 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.295952    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlrm8\" (UniqueName: \"kubernetes.io/projected/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541-kube-api-access-xlrm8\") pod \"payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp\" (UID: \"9c7d9808-fd6c-480c-8b6a-1f3a47eb6541\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"
Oct 30 21:45:51.296183 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.296041    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp\" (UID: \"9c7d9808-fd6c-480c-8b6a-1f3a47eb6541\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"
Oct 30 21:45:51.296893 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.296866    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp\" (UID: \"9c7d9808-fd6c-480c-8b6a-1f3a47eb6541\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"
Oct 30 21:45:51.352581 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.352545    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlrm8\" (UniqueName: \"kubernetes.io/projected/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541-kube-api-access-xlrm8\") pod \"payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp\" (UID: \"9c7d9808-fd6c-480c-8b6a-1f3a47eb6541\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"
Oct 30 21:45:51.377556 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.377526    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"
Oct 30 21:45:51.480688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.480643    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"
Oct 30 21:45:51.755098 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.755056    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"]
Oct 30 21:45:51.756953 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:45:51.756923    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33ecbfc9_fecd_4f7a_b162_40bd00a040b9.slice/crio-4421b84d7dcd73739e62caebf3573adf5437a77524341586234b946077b93743 WatchSource:0}: Error finding container 4421b84d7dcd73739e62caebf3573adf5437a77524341586234b946077b93743: Status 404 returned error can't find the container with id 4421b84d7dcd73739e62caebf3573adf5437a77524341586234b946077b93743
Oct 30 21:45:51.866732 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:51.866688    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"]
Oct 30 21:45:51.869422 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:45:51.869392    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c7d9808_fd6c_480c_8b6a_1f3a47eb6541.slice/crio-9d70a304e59596ee357121bd0742174e2359198130572018fc9a33ca563ce454 WatchSource:0}: Error finding container 9d70a304e59596ee357121bd0742174e2359198130572018fc9a33ca563ce454: Status 404 returned error can't find the container with id 9d70a304e59596ee357121bd0742174e2359198130572018fc9a33ca563ce454
Oct 30 21:45:52.191001 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:52.190965    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h" event={"ID":"33ecbfc9-fecd-4f7a-b162-40bd00a040b9","Type":"ContainerStarted","Data":"753c02f3f27330cd76522e3a763ae6ba68e6abae22b343dfb145e4e48260965d"}
Oct 30 21:45:52.191475 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:52.191030    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h" event={"ID":"33ecbfc9-fecd-4f7a-b162-40bd00a040b9","Type":"ContainerStarted","Data":"4421b84d7dcd73739e62caebf3573adf5437a77524341586234b946077b93743"}
Oct 30 21:45:52.191475 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:52.191136    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"
Oct 30 21:45:52.192666 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:52.192640    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp" event={"ID":"9c7d9808-fd6c-480c-8b6a-1f3a47eb6541","Type":"ContainerStarted","Data":"86c009b69912dff0354810d1be4e8fb3cc4651b30d9ef4dd2b0cf716f65873fc"}
Oct 30 21:45:52.192782 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:52.192685    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp" event={"ID":"9c7d9808-fd6c-480c-8b6a-1f3a47eb6541","Type":"ContainerStarted","Data":"9d70a304e59596ee357121bd0742174e2359198130572018fc9a33ca563ce454"}
Oct 30 21:45:52.192782 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:52.192712    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"
Oct 30 21:45:52.304637 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:52.304578    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h" podStartSLOduration=1.304564926 podStartE2EDuration="1.304564926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:52.303209844 +0000 UTC m=+694.975113780" watchObservedRunningTime="2026-10-30 21:45:52.304564926 +0000 UTC m=+694.976468910"
Oct 30 21:45:52.424723 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:52.424667    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp" podStartSLOduration=1.4246517650000001 podStartE2EDuration="1.424651765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:52.421525398 +0000 UTC m=+695.093429332" watchObservedRunningTime="2026-10-30 21:45:52.424651765 +0000 UTC m=+695.096555698"
Oct 30 21:45:53.156559 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.156524    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:53.335869 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.335825    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:53.557364 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.557284    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"]
Oct 30 21:45:53.561039 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.561023    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"
Oct 30 21:45:53.617650 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.617618    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"]
Oct 30 21:45:53.622569 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.622538    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fe6d48f-81e8-4900-8249-0a73a66fc86b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48\" (UID: \"1fe6d48f-81e8-4900-8249-0a73a66fc86b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"
Oct 30 21:45:53.622768 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.622585    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhgfs\" (UniqueName: \"kubernetes.io/projected/1fe6d48f-81e8-4900-8249-0a73a66fc86b-kube-api-access-zhgfs\") pod \"payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48\" (UID: \"1fe6d48f-81e8-4900-8249-0a73a66fc86b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"
Oct 30 21:45:53.663974 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.663941    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-80c366-dockercfg-s6n9d\""
Oct 30 21:45:53.663974 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.663941    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-80c366\""
Oct 30 21:45:53.719183 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.719148    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"]
Oct 30 21:45:53.723517 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.723483    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fe6d48f-81e8-4900-8249-0a73a66fc86b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48\" (UID: \"1fe6d48f-81e8-4900-8249-0a73a66fc86b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"
Oct 30 21:45:53.723670 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.723534    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhgfs\" (UniqueName: \"kubernetes.io/projected/1fe6d48f-81e8-4900-8249-0a73a66fc86b-kube-api-access-zhgfs\") pod \"payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48\" (UID: \"1fe6d48f-81e8-4900-8249-0a73a66fc86b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"
Oct 30 21:45:53.723670 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.723627    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"
Oct 30 21:45:53.724569 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.724545    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fe6d48f-81e8-4900-8249-0a73a66fc86b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48\" (UID: \"1fe6d48f-81e8-4900-8249-0a73a66fc86b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"
Oct 30 21:45:53.757738 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.757697    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"]
Oct 30 21:45:53.783039 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.783007    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhgfs\" (UniqueName: \"kubernetes.io/projected/1fe6d48f-81e8-4900-8249-0a73a66fc86b-kube-api-access-zhgfs\") pod \"payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48\" (UID: \"1fe6d48f-81e8-4900-8249-0a73a66fc86b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"
Oct 30 21:45:53.824224 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.824124    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrkrw\" (UniqueName: \"kubernetes.io/projected/ede829db-672f-49fa-9b86-00d130c21560-kube-api-access-rrkrw\") pod \"payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g\" (UID: \"ede829db-672f-49fa-9b86-00d130c21560\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"
Oct 30 21:45:53.824419 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.824280    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ede829db-672f-49fa-9b86-00d130c21560-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g\" (UID: \"ede829db-672f-49fa-9b86-00d130c21560\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"
Oct 30 21:45:53.874181 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.874147    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"
Oct 30 21:45:53.925031 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.924998    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrkrw\" (UniqueName: \"kubernetes.io/projected/ede829db-672f-49fa-9b86-00d130c21560-kube-api-access-rrkrw\") pod \"payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g\" (UID: \"ede829db-672f-49fa-9b86-00d130c21560\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"
Oct 30 21:45:53.925271 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.925087    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ede829db-672f-49fa-9b86-00d130c21560-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g\" (UID: \"ede829db-672f-49fa-9b86-00d130c21560\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"
Oct 30 21:45:53.926635 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.925998    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ede829db-672f-49fa-9b86-00d130c21560-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g\" (UID: \"ede829db-672f-49fa-9b86-00d130c21560\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"
Oct 30 21:45:53.983574 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:53.983540    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrkrw\" (UniqueName: \"kubernetes.io/projected/ede829db-672f-49fa-9b86-00d130c21560-kube-api-access-rrkrw\") pod \"payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g\" (UID: \"ede829db-672f-49fa-9b86-00d130c21560\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"
Oct 30 21:45:54.038575 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:54.038548    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"
Oct 30 21:45:54.252697 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:54.252663    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"]
Oct 30 21:45:54.412351 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:54.412328    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"]
Oct 30 21:45:54.413964 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:45:54.413941    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podede829db_672f_49fa_9b86_00d130c21560.slice/crio-3b405133446a7b8bd6c479be252950c8271beea2bd1ae74597d9f15ed10b8d48 WatchSource:0}: Error finding container 3b405133446a7b8bd6c479be252950c8271beea2bd1ae74597d9f15ed10b8d48: Status 404 returned error can't find the container with id 3b405133446a7b8bd6c479be252950c8271beea2bd1ae74597d9f15ed10b8d48
Oct 30 21:45:55.211245 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:55.211189    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48" event={"ID":"1fe6d48f-81e8-4900-8249-0a73a66fc86b","Type":"ContainerStarted","Data":"efc2e975250cfaedd0ce1f11668076a2d16f9e1a29a18b20cf8fee9169242e13"}
Oct 30 21:45:55.211430 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:55.211404    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"
Oct 30 21:45:55.211512 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:55.211450    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48" event={"ID":"1fe6d48f-81e8-4900-8249-0a73a66fc86b","Type":"ContainerStarted","Data":"17f77054f542f04d966240ea133007a72ab11e9c06f538949a15d221c7a15f2a"}
Oct 30 21:45:55.212854 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:55.212831    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g" event={"ID":"ede829db-672f-49fa-9b86-00d130c21560","Type":"ContainerStarted","Data":"98dd6319631f816a0f877ad3a18a563734979b0ab0dadee09dd0b36ba8d08fd0"}
Oct 30 21:45:55.212984 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:55.212864    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g" event={"ID":"ede829db-672f-49fa-9b86-00d130c21560","Type":"ContainerStarted","Data":"3b405133446a7b8bd6c479be252950c8271beea2bd1ae74597d9f15ed10b8d48"}
Oct 30 21:45:55.213040 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:55.212981    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"
Oct 30 21:45:55.326341 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:55.326278    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48" podStartSLOduration=2.326227152 podStartE2EDuration="2.326227152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:55.324655987 +0000 UTC m=+697.996559922" watchObservedRunningTime="2026-10-30 21:45:55.326227152 +0000 UTC m=+697.998131086"
Oct 30 21:45:55.447567 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:55.447512    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g" podStartSLOduration=2.447486809 podStartE2EDuration="2.447486809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:55.442435045 +0000 UTC m=+698.114338981" watchObservedRunningTime="2026-10-30 21:45:55.447486809 +0000 UTC m=+698.119390793"
Oct 30 21:45:55.618180 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:55.618091    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:55.817483 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:55.817448    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:55.983591 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:55.983553    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:55.998794 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:55.998760    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"]
Oct 30 21:45:56.002858 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:56.002839    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"
Oct 30 21:45:56.047615 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:56.047577    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7pvw\" (UniqueName: \"kubernetes.io/projected/56ed6524-65dc-4fe7-bebc-cdd266746388-kube-api-access-m7pvw\") pod \"payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b\" (UID: \"56ed6524-65dc-4fe7-bebc-cdd266746388\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"
Oct 30 21:45:56.047771 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:56.047643    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ed6524-65dc-4fe7-bebc-cdd266746388-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b\" (UID: \"56ed6524-65dc-4fe7-bebc-cdd266746388\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"
Oct 30 21:45:56.056615 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:56.056556    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-bc009f\""
Oct 30 21:45:56.056826 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:56.056738    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-bc009f-dockercfg-s5llr\""
Oct 30 21:45:56.060080 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:56.060053    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"]
Oct 30 21:45:56.148678 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:56.148643    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ed6524-65dc-4fe7-bebc-cdd266746388-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b\" (UID: \"56ed6524-65dc-4fe7-bebc-cdd266746388\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"
Oct 30 21:45:56.148862 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:56.148745    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7pvw\" (UniqueName: \"kubernetes.io/projected/56ed6524-65dc-4fe7-bebc-cdd266746388-kube-api-access-m7pvw\") pod \"payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b\" (UID: \"56ed6524-65dc-4fe7-bebc-cdd266746388\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"
Oct 30 21:45:56.149337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:56.149317    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ed6524-65dc-4fe7-bebc-cdd266746388-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b\" (UID: \"56ed6524-65dc-4fe7-bebc-cdd266746388\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"
Oct 30 21:45:56.160798 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:56.160767    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:56.303711 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:56.290995    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7pvw\" (UniqueName: \"kubernetes.io/projected/56ed6524-65dc-4fe7-bebc-cdd266746388-kube-api-access-m7pvw\") pod \"payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b\" (UID: \"56ed6524-65dc-4fe7-bebc-cdd266746388\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"
Oct 30 21:45:56.317059 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:56.317024    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"
Oct 30 21:45:56.337819 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:56.337790    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:56.693584 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:56.693549    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"]
Oct 30 21:45:56.695386 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:45:56.695356    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56ed6524_65dc_4fe7_bebc_cdd266746388.slice/crio-6409d503bb3bdca429d88151a3fad3b87eaa39ab5c0e1073b515a631bc77d406 WatchSource:0}: Error finding container 6409d503bb3bdca429d88151a3fad3b87eaa39ab5c0e1073b515a631bc77d406: Status 404 returned error can't find the container with id 6409d503bb3bdca429d88151a3fad3b87eaa39ab5c0e1073b515a631bc77d406
Oct 30 21:45:57.262089 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:57.262051    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b" event={"ID":"56ed6524-65dc-4fe7-bebc-cdd266746388","Type":"ContainerStarted","Data":"8c4c01ae8471b03b19adaab304af8e0ebeca4bd5d9872eca3a3f2659efeee649"}
Oct 30 21:45:57.262283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:57.262121    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"
Oct 30 21:45:57.262283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:57.262145    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b" event={"ID":"56ed6524-65dc-4fe7-bebc-cdd266746388","Type":"ContainerStarted","Data":"6409d503bb3bdca429d88151a3fad3b87eaa39ab5c0e1073b515a631bc77d406"}
Oct 30 21:45:57.378771 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:57.378694    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b" podStartSLOduration=2.378680306 podStartE2EDuration="2.378680306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:57.377001661 +0000 UTC m=+700.048905595" watchObservedRunningTime="2026-10-30 21:45:57.378680306 +0000 UTC m=+700.050584218"
Oct 30 21:45:58.165399 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:58.165367    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:58.344550 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:45:58.344510    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:00.313331 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:00.313278    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:00.547437 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:00.543194    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:00.779326 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:00.779289    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:00.965270 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:00.964840    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:01.193111 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:01.193064    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"
Oct 30 21:46:01.198249 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:01.198210    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"
Oct 30 21:46:01.370770 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:01.370738    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:01.557207 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:01.557109    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:01.714621 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:01.714588    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:03.201382 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:03.201357    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"
Oct 30 21:46:03.201876 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:03.201861    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"
Oct 30 21:46:04.946970 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:04.946927    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:05.107556 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.107516    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:05.341779 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.341610    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"]
Oct 30 21:46:05.350253 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.350115    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"
Oct 30 21:46:05.408217 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.408181    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"]
Oct 30 21:46:05.457322 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.457288    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-c3e2d8\""
Oct 30 21:46:05.457760 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.457729    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-c3e2d8-dockercfg-gjqsk\""
Oct 30 21:46:05.509255 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.509200    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"]
Oct 30 21:46:05.513068 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.513041    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"
Oct 30 21:46:05.551312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.551276    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"]
Oct 30 21:46:05.558804 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.558742    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daf26fab-89e6-4d8d-ad82-0919c933c32a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf\" (UID: \"daf26fab-89e6-4d8d-ad82-0919c933c32a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"
Oct 30 21:46:05.559013 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.558979    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-697vx\" (UniqueName: \"kubernetes.io/projected/daf26fab-89e6-4d8d-ad82-0919c933c32a-kube-api-access-697vx\") pod \"payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf\" (UID: \"daf26fab-89e6-4d8d-ad82-0919c933c32a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"
Oct 30 21:46:05.659943 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.659902    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnkm6\" (UniqueName: \"kubernetes.io/projected/556f26e0-ea90-47c8-af47-19814cb09daa-kube-api-access-mnkm6\") pod \"payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5\" (UID: \"556f26e0-ea90-47c8-af47-19814cb09daa\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"
Oct 30 21:46:05.660154 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.659960    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daf26fab-89e6-4d8d-ad82-0919c933c32a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf\" (UID: \"daf26fab-89e6-4d8d-ad82-0919c933c32a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"
Oct 30 21:46:05.660154 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.660135    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/556f26e0-ea90-47c8-af47-19814cb09daa-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5\" (UID: \"556f26e0-ea90-47c8-af47-19814cb09daa\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"
Oct 30 21:46:05.660286 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.660270    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-697vx\" (UniqueName: \"kubernetes.io/projected/daf26fab-89e6-4d8d-ad82-0919c933c32a-kube-api-access-697vx\") pod \"payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf\" (UID: \"daf26fab-89e6-4d8d-ad82-0919c933c32a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"
Oct 30 21:46:05.660936 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.660908    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daf26fab-89e6-4d8d-ad82-0919c933c32a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf\" (UID: \"daf26fab-89e6-4d8d-ad82-0919c933c32a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"
Oct 30 21:46:05.717422 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.717387    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-697vx\" (UniqueName: \"kubernetes.io/projected/daf26fab-89e6-4d8d-ad82-0919c933c32a-kube-api-access-697vx\") pod \"payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf\" (UID: \"daf26fab-89e6-4d8d-ad82-0919c933c32a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"
Oct 30 21:46:05.761592 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.761541    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/556f26e0-ea90-47c8-af47-19814cb09daa-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5\" (UID: \"556f26e0-ea90-47c8-af47-19814cb09daa\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"
Oct 30 21:46:05.761779 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.761654    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnkm6\" (UniqueName: \"kubernetes.io/projected/556f26e0-ea90-47c8-af47-19814cb09daa-kube-api-access-mnkm6\") pod \"payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5\" (UID: \"556f26e0-ea90-47c8-af47-19814cb09daa\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"
Oct 30 21:46:05.762677 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.762645    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/556f26e0-ea90-47c8-af47-19814cb09daa-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5\" (UID: \"556f26e0-ea90-47c8-af47-19814cb09daa\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"
Oct 30 21:46:05.818241 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.818196    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnkm6\" (UniqueName: \"kubernetes.io/projected/556f26e0-ea90-47c8-af47-19814cb09daa-kube-api-access-mnkm6\") pod \"payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5\" (UID: \"556f26e0-ea90-47c8-af47-19814cb09daa\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"
Oct 30 21:46:05.842710 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.842677    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"
Oct 30 21:46:05.964019 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:05.963924    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"
Oct 30 21:46:06.223454 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:06.223418    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"]
Oct 30 21:46:06.224131 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:46:06.224100    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod556f26e0_ea90_47c8_af47_19814cb09daa.slice/crio-fc2f98057a38a35be85d7560184de44f786c6f5b67d150b089a3bbad3abb2b54 WatchSource:0}: Error finding container fc2f98057a38a35be85d7560184de44f786c6f5b67d150b089a3bbad3abb2b54: Status 404 returned error can't find the container with id fc2f98057a38a35be85d7560184de44f786c6f5b67d150b089a3bbad3abb2b54
Oct 30 21:46:06.258417 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:06.258390    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"
Oct 30 21:46:06.258586 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:06.258458    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"
Oct 30 21:46:06.323655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:06.323607    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5" event={"ID":"556f26e0-ea90-47c8-af47-19814cb09daa","Type":"ContainerStarted","Data":"821269e7fce1eb7831ec49c04cb5ab03b10f342d7afa3c168daffdd01d69f509"}
Oct 30 21:46:06.323836 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:06.323682    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5" event={"ID":"556f26e0-ea90-47c8-af47-19814cb09daa","Type":"ContainerStarted","Data":"fc2f98057a38a35be85d7560184de44f786c6f5b67d150b089a3bbad3abb2b54"}
Oct 30 21:46:06.323909 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:06.323854    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"
Oct 30 21:46:06.342068 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:06.342042    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"]
Oct 30 21:46:06.344217 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:46:06.344193    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaf26fab_89e6_4d8d_ad82_0919c933c32a.slice/crio-5ea38fcd22e44af4eb76156d7a86de030939b668dcfd7c537ecbf499a3c14392 WatchSource:0}: Error finding container 5ea38fcd22e44af4eb76156d7a86de030939b668dcfd7c537ecbf499a3c14392: Status 404 returned error can't find the container with id 5ea38fcd22e44af4eb76156d7a86de030939b668dcfd7c537ecbf499a3c14392
Oct 30 21:46:06.600171 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:06.600057    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5" podStartSLOduration=1.600038154 podStartE2EDuration="1.600038154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:06.598985504 +0000 UTC m=+709.270889440" watchObservedRunningTime="2026-10-30 21:46:06.600038154 +0000 UTC m=+709.271942090"
Oct 30 21:46:06.905755 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:06.905708    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:07.331529 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:07.331431    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf" event={"ID":"daf26fab-89e6-4d8d-ad82-0919c933c32a","Type":"ContainerStarted","Data":"90083aeb46c6356fe6f1cafc2a553acc7c5842aade470a5642aaa7718ff29b56"}
Oct 30 21:46:07.331529 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:07.331496    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf" event={"ID":"daf26fab-89e6-4d8d-ad82-0919c933c32a","Type":"ContainerStarted","Data":"5ea38fcd22e44af4eb76156d7a86de030939b668dcfd7c537ecbf499a3c14392"}
Oct 30 21:46:07.331966 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:07.331673    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"
Oct 30 21:46:07.449400 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:07.449349    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf" podStartSLOduration=2.449333397 podStartE2EDuration="2.449333397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:07.446542331 +0000 UTC m=+710.118446269" watchObservedRunningTime="2026-10-30 21:46:07.449333397 +0000 UTC m=+710.121237332"
Oct 30 21:46:08.272789 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:08.272753    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"
Oct 30 21:46:09.346098 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:09.346061    5586 generic.go:356] "Generic (PLEG): container finished" podID="37583645-5073-494b-8079-7eac9ceb5d30" containerID="1e5933086ae37e06e5887a52e8c3336a2e5e5992bbca81f0c5c22f870da44bb0" exitCode=0
Oct 30 21:46:09.346582 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:09.346135    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf" event={"ID":"37583645-5073-494b-8079-7eac9ceb5d30","Type":"ContainerDied","Data":"1e5933086ae37e06e5887a52e8c3336a2e5e5992bbca81f0c5c22f870da44bb0"}
Oct 30 21:46:09.346626 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:09.346600    5586 scope.go:122] "RemoveContainer" containerID="1e5933086ae37e06e5887a52e8c3336a2e5e5992bbca81f0c5c22f870da44bb0"
Oct 30 21:46:09.347905 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:09.347876    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:46:09.347990 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:09.347945    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:46:09.347990 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:09.347975    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:46:10.354171 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:10.354133    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf" event={"ID":"37583645-5073-494b-8079-7eac9ceb5d30","Type":"ContainerStarted","Data":"595518896f12d3881879831fe0a1ad923f36e5ff30b6e49ea1f59034cb47e9cd"}
Oct 30 21:46:10.806594 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:10.806552    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:46:10.807917 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:10.807893    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Oct 30 21:46:10.807990 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:10.807943    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf" podUID="37583645-5073-494b-8079-7eac9ceb5d30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Oct 30 21:46:11.806708 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:11.806661    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Oct 30 21:46:11.807202 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:11.806739    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf" podUID="37583645-5073-494b-8079-7eac9ceb5d30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Oct 30 21:46:11.947914 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:11.947872    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:12.094152 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:12.094064    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:46:12.102967 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:12.102932    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:12.259252 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:12.259198    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:12.806767 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:12.806728    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Oct 30 21:46:12.807191 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:12.806786    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf" podUID="37583645-5073-494b-8079-7eac9ceb5d30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Oct 30 21:46:13.806363 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:13.806327    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Oct 30 21:46:13.806571 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:13.806391    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf" podUID="37583645-5073-494b-8079-7eac9ceb5d30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Oct 30 21:46:14.805815 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:14.805780    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Oct 30 21:46:14.806227 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:14.805834    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf" podUID="37583645-5073-494b-8079-7eac9ceb5d30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Oct 30 21:46:15.810152 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:15.810119    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:46:15.811274 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:15.811255    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 21:46:17.104765 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:17.104728    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:17.334978 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:17.334938    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"
Oct 30 21:46:18.342559 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:18.342528    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"
Oct 30 21:46:18.375018 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:46:18.374986    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:46:19.200060 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:19.200015    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:22.047061 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:22.047009    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:22.283179 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:22.283140    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:22.495353 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:22.495299    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:24.620472 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:24.620427    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:25.203056 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:25.203023    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:25.416272 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:25.416163    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:27.240508 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:27.240465    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:27.481651 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:27.481615    5586 generic.go:356] "Generic (PLEG): container finished" podID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerID="67efb1ff70be31453fb0b03d24f694c7cf42d57cd50d0effc4a938b0eaf29e08" exitCode=0
Oct 30 21:46:27.481863 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:27.481695    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" event={"ID":"b40a8985-4520-4dbc-894d-b07e94d9deac","Type":"ContainerDied","Data":"67efb1ff70be31453fb0b03d24f694c7cf42d57cd50d0effc4a938b0eaf29e08"}
Oct 30 21:46:27.482145 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:27.482125    5586 scope.go:122] "RemoveContainer" containerID="67efb1ff70be31453fb0b03d24f694c7cf42d57cd50d0effc4a938b0eaf29e08"
Oct 30 21:46:27.483624 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:27.483598    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:46:27.483700 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:27.483668    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:46:27.483740 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:27.483698    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:46:27.566253 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:27.564114    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:27.836005 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:27.835896    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:28.110226 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:28.105917    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:46:28.489424 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:28.489350    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" event={"ID":"b40a8985-4520-4dbc-894d-b07e94d9deac","Type":"ContainerStarted","Data":"a33ea16f9a593ba8a1b8397b2571f32e61fff0cb063262db54ac27938a9dcf05"}
Oct 30 21:46:29.405636 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:29.405602    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:46:29.406710 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:29.406686    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:46:29.406771 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:29.406733    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 21:46:29.795211 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:29.795073    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:30.405212 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:30.405178    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:46:30.405419 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:30.405264    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 21:46:31.405041 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:31.404998    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:46:31.405534 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:31.405071    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 21:46:32.013084 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.013040    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"]
Oct 30 21:46:32.016118 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.016093    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:32.073258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.072736    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:46:32.082280 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.080126    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:46:32.110775 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.110738    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-metrics-certs\") pod \"maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:32.111003 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.110989    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-maas-api-tls\") pod \"maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:32.111280 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.111130    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b85c642-f2d3-4fb1-9328-3dea529752de-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:32.111280 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.111197    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2zb2\" (UniqueName: \"kubernetes.io/projected/6b85c642-f2d3-4fb1-9328-3dea529752de-kube-api-access-r2zb2\") pod \"maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:32.177276 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.169545    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fqgfc\""
Oct 30 21:46:32.180760 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.179438    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-dda9f5-serving-cert\""
Oct 30 21:46:32.180760 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.179448    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 21:46:32.180760 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.178898    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"]
Oct 30 21:46:32.213048 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.213004    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2zb2\" (UniqueName: \"kubernetes.io/projected/6b85c642-f2d3-4fb1-9328-3dea529752de-kube-api-access-r2zb2\") pod \"maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:32.213253 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.213114    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-metrics-certs\") pod \"maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:32.213253 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.213139    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-maas-api-tls\") pod \"maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:32.213253 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.213175    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b85c642-f2d3-4fb1-9328-3dea529752de-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:32.213821 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.213800    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b85c642-f2d3-4fb1-9328-3dea529752de-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:32.213910 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:46:32.213897    5586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-serving-cert: secret "maas-api-e2e-worker-w0-dda9f5-serving-cert" not found
Oct 30 21:46:32.213975 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:46:32.213967    5586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-maas-api-tls podName:6b85c642-f2d3-4fb1-9328-3dea529752de nodeName:}" failed. No retries permitted until 2026-10-30 21:46:32.713952111 +0000 UTC m=+735.385856025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-maas-api-tls") pod "maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx" (UID: "6b85c642-f2d3-4fb1-9328-3dea529752de") : secret "maas-api-e2e-worker-w0-dda9f5-serving-cert" not found
Oct 30 21:46:32.216099 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.216073    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-metrics-certs\") pod \"maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:32.283781 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.283681    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:32.290024 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.289993    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2zb2\" (UniqueName: \"kubernetes.io/projected/6b85c642-f2d3-4fb1-9328-3dea529752de-kube-api-access-r2zb2\") pod \"maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:32.405095 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.405053    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:46:32.405539 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.405143    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 21:46:32.444262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.443203    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:32.597184 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.597099    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:32.719673 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.719643    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-maas-api-tls\") pod \"maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:32.751912 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.751878    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-maas-api-tls\") pod \"maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:32.970580 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:32.970526    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:33.385694 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:33.385655    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"]
Oct 30 21:46:33.404937 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:33.404904    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:46:33.405107 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:33.404954    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 21:46:33.519909 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:33.519872    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx" event={"ID":"6b85c642-f2d3-4fb1-9328-3dea529752de","Type":"ContainerStarted","Data":"5daa538e03fb0aa717a38aff1fe772c7606f0eb8a7d6df44912b35f25417f10a"}
Oct 30 21:46:34.409438 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:34.409400    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:46:34.409846 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:34.409829    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:46:34.410651 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:34.410631    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 21:46:36.544143 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:36.544091    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx" event={"ID":"6b85c642-f2d3-4fb1-9328-3dea529752de","Type":"ContainerStarted","Data":"8f56572cff08c4306612591c7fbc05a38ec653c060334baed9f59d2c8a84ea60"}
Oct 30 21:46:36.544697 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:36.544200    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:36.659744 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:36.659683    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx" podStartSLOduration=3.33574667 podStartE2EDuration="5.659663156s" totalImagesPullingTime="2.323916486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:36.658794099 +0000 UTC m=+739.330698047" watchObservedRunningTime="2026-10-30 21:46:36.659663156 +0000 UTC m=+739.331567094"
Oct 30 21:46:37.174226 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:37.174175    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:38.560352 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:38.560316    5586 generic.go:356] "Generic (PLEG): container finished" podID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerID="803b026b4bc0d8aa4d34faa0bef1c98e42c98b35add07399fd76f20e4bf51ab3" exitCode=0
Oct 30 21:46:38.560947 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:38.560357    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd" event={"ID":"cd9299f1-9812-4f6c-8851-6957b8b3ab64","Type":"ContainerDied","Data":"803b026b4bc0d8aa4d34faa0bef1c98e42c98b35add07399fd76f20e4bf51ab3"}
Oct 30 21:46:38.560947 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:38.560813    5586 scope.go:122] "RemoveContainer" containerID="803b026b4bc0d8aa4d34faa0bef1c98e42c98b35add07399fd76f20e4bf51ab3"
Oct 30 21:46:38.562467 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:38.562430    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:46:38.562579 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:38.562528    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:46:38.562624 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:38.562584    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:46:39.575723 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:39.575677    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd" event={"ID":"cd9299f1-9812-4f6c-8851-6957b8b3ab64","Type":"ContainerStarted","Data":"49626e30c184a07ec9f242c441f2cad15c993f1b4e120cf51fafa38ac77ad15c"}
Oct 30 21:46:39.857108 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:39.856987    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 21:46:39.857440 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:39.857397    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd" podUID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 21:46:39.857990 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:39.855566    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:46:40.851169 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:40.850684    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 21:46:40.851169 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:40.850754    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd" podUID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 21:46:41.851269 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:41.850657    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 21:46:41.851269 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:41.850735    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd" podUID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 21:46:42.094283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:42.094221    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:46:42.253222 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:42.253186    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:42.571935 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:42.571841    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 21:46:42.851318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:42.850644    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 21:46:42.851318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:42.850722    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd" podUID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 21:46:43.853760 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:43.853723    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 21:46:43.854383 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:43.854350    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd" podUID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 21:46:44.850533 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:44.850489    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 21:46:44.850731 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:44.850566    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd" podUID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 21:46:45.861169 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:45.859930    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:46:45.861752 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:45.861733    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 21:46:45.867378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:45.867331    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:46.465023 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:46.464982    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:46.634750 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:46.634707    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:46.802947 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:46.802862    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:47.534098 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:47.534058    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:48.361456 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:46:48.361347    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:46:48.635956 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:48.635873    5586 generic.go:356] "Generic (PLEG): container finished" podID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerID="13a09357e9e29b57a29134c2c276d66a096fddf2e9d2aa3d197a7cd71499f430" exitCode=0
Oct 30 21:46:48.635956 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:48.635941    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5" event={"ID":"93475d2f-6f42-47e4-9c1e-44cd9080924c","Type":"ContainerDied","Data":"13a09357e9e29b57a29134c2c276d66a096fddf2e9d2aa3d197a7cd71499f430"}
Oct 30 21:46:48.636483 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:48.636373    5586 scope.go:122] "RemoveContainer" containerID="13a09357e9e29b57a29134c2c276d66a096fddf2e9d2aa3d197a7cd71499f430"
Oct 30 21:46:48.637756 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:48.637731    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:46:48.637825 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:48.637815    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:46:48.637860 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:48.637848    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:46:49.667285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:49.656997    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5" event={"ID":"93475d2f-6f42-47e4-9c1e-44cd9080924c","Type":"ContainerStarted","Data":"abe8dcf371ee3779ae62c82677519870e0d0a21a296f068db885f886606abf8c"}
Oct 30 21:46:49.795255 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:49.791687    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:46:49.795255 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:49.791827    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:46:49.795255 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:49.791872    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5" podUID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 21:46:50.161257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:50.160890    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:50.746263 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:50.745850    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:50.790117 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:50.790081    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:46:50.790343 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:50.790153    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5" podUID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 21:46:50.906413 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:50.906367    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:51.789599 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:51.789553    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:46:51.790113 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:51.789628    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5" podUID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 21:46:52.789533 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:52.789489    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:46:52.789735 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:52.789563    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5" podUID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 21:46:52.857315 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:52.851430    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:53.790025 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:53.789982    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:46:53.790475 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:53.790040    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5" podUID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 21:46:54.790000 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:54.789954    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:46:54.790196 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:54.790031    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5" podUID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 21:46:55.794295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:55.794262    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:46:55.794825 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:55.794693    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:46:55.795564 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:55.795543    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 21:46:57.599675 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:57.599580    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:57.763963 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:46:57.763925    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:02.705182 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:02.705140    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:07.593354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:07.593303    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:10.184182 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:10.184142    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:10.841950 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:10.841916    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:11.068151 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:11.068114    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:11.585305 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:11.585265    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:12.274860 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:12.274825    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:12.457202 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:12.457157    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:13.279319 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:13.279281    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:14.388215 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:14.388180    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:18.364030 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:47:18.363986    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:47:18.583051 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:18.583021    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:19.527648 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:19.527612    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:22.725069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:22.725031    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:23.421216 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:23.421182    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:23.582576 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:23.582520    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:23.766664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:23.766530    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:25.594411 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:25.594374    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:27.168303 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:27.168264    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:27.919718 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:27.919679    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:28.112246 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:28.112207    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:28.315133 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:28.315037    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:29.016327 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:29.016294    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:31.917995 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:31.913666    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:34.087618 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:34.087587    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:38.894979 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:38.894877    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:43.839668 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:43.839622    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:47.082467 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:47.082429    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:48.361617 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:47:48.361580    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:47:48.888562 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:48.888519    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:53.863142 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:53.863105    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:58.611110 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:58.611015    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:58.812923 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:47:58.812886    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:03.808860 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:03.808826    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:05.834805 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:05.834753    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:08.926259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:08.922464    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:14.122337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.122296    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:14.274339 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.274303    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"]
Oct 30 21:48:14.278555 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.278534    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.378762 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.378673    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-tmp-dir\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.378762 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.378736    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-model-cache\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.378958 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.378803    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5nbs\" (UniqueName: \"kubernetes.io/projected/e920b103-cdeb-4a9a-ac7f-447d89d5033f-kube-api-access-b5nbs\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.378958 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.378846    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-home\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.379029 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.378985    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-dshm\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.379029 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.379008    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e920b103-cdeb-4a9a-ac7f-447d89d5033f-tls-certs\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.381679 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.381657    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-80c366\"/\"premium-w1-80c366-kserve-self-signed-certs\""
Oct 30 21:48:14.381871 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.381858    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-80c366\"/\"default-dockercfg-xrtxm\""
Oct 30 21:48:14.381918 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.381867    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-80c366\"/\"kube-root-ca.crt\""
Oct 30 21:48:14.381918 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.381872    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-80c366\"/\"openshift-service-ca.crt\""
Oct 30 21:48:14.384438 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.384417    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"]
Oct 30 21:48:14.389899 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.389863    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:14.480448 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.480411    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-dshm\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.480714 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.480605    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e920b103-cdeb-4a9a-ac7f-447d89d5033f-tls-certs\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.480714 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.480697    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-tmp-dir\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.480844 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.480790    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-model-cache\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.480844 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.480820    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5nbs\" (UniqueName: \"kubernetes.io/projected/e920b103-cdeb-4a9a-ac7f-447d89d5033f-kube-api-access-b5nbs\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.480952 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.480888    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-home\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.481042 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.481022    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-tmp-dir\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.481124 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.481102    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-model-cache\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.481281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.481260    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-home\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.482688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.482669    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-dshm\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.482995 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.482978    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e920b103-cdeb-4a9a-ac7f-447d89d5033f-tls-certs\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.542610 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.542579    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5nbs\" (UniqueName: \"kubernetes.io/projected/e920b103-cdeb-4a9a-ac7f-447d89d5033f-kube-api-access-b5nbs\") pod \"premium-w1-80c366-kserve-66b94476f9-hgms6\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") " pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.590845 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.590808    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:14.999658 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:14.999627    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"]
Oct 30 21:48:15.000729 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:48:15.000702    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode920b103_cdeb_4a9a_ac7f_447d89d5033f.slice/crio-0a4089870469d01048069e5245b75121d267f47797916ef9c07bbb04bd045b58 WatchSource:0}: Error finding container 0a4089870469d01048069e5245b75121d267f47797916ef9c07bbb04bd045b58: Status 404 returned error can't find the container with id 0a4089870469d01048069e5245b75121d267f47797916ef9c07bbb04bd045b58
Oct 30 21:48:15.209529 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:15.209490    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" event={"ID":"e920b103-cdeb-4a9a-ac7f-447d89d5033f","Type":"ContainerStarted","Data":"662c4cf2dc22586d03f2a204406b8c712e1090ca4a0403b402301b39d90d0448"}
Oct 30 21:48:15.209901 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:15.209542    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" event={"ID":"e920b103-cdeb-4a9a-ac7f-447d89d5033f","Type":"ContainerStarted","Data":"0a4089870469d01048069e5245b75121d267f47797916ef9c07bbb04bd045b58"}
Oct 30 21:48:15.327543 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:15.327435    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podStartSLOduration=1.327420378 podStartE2EDuration="1.327420378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:48:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:48:15.326806464 +0000 UTC m=+837.998710400" watchObservedRunningTime="2026-10-30 21:48:15.327420378 +0000 UTC m=+837.999324313"
Oct 30 21:48:15.591687 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:15.591591    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:18.360813 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:48:18.360773    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:48:19.203795 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:19.203760    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:19.887103 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:19.887065    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:23.980104 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:23.980069    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:24.591209 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:24.591155    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:24.604068 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:24.604037    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:25.284496 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:25.284458    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:48:28.885461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:28.885425    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:29.664109 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:29.664069    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:33.700289 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:33.700224    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:38.494296 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:38.494260    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:43.284438 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:43.284394    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:48.188701 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:48.188660    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:48.361174 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:48:48.361138    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:48:49.556608 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:49.556569    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:49.712516 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:49.712479    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:53.109148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:53.108651    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:57.853378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:57.853338    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:58.063519 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:48:58.063485    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:02.779584 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:02.779552    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:07.587941 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:07.587881    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:08.574484 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:08.574450    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:12.586824 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:12.586760    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:14.385929 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:14.385888    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:14.587563 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:14.587508    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:17.393807 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:17.393778    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:17.565143 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:17.565113    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 21:49:17.565347 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:17.565172    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 21:49:17.570639 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:17.570615    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 21:49:17.570807 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:17.570618    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 21:49:18.361500 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:49:18.361457    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:49:20.227739 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:20.227695    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:22.195210 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:22.195172    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:26.937213 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:26.933322    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"]
Oct 30 21:49:26.939215 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:26.939187    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:26.987244 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:26.987192    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:26.994425 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:26.994398    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"]
Oct 30 21:49:27.018805 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.018762    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wmzk\" (UniqueName: \"kubernetes.io/projected/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-kube-api-access-8wmzk\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.018805 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.018813    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-dshm\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.018998 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.018962    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-home\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.018998 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.018991    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-tmp-dir\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.019102 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.019088    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-model-cache\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.019149 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.019136    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-tls-certs\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.041108 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.041074    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-dda9f5\"/\"default-dockercfg-47mqx\""
Oct 30 21:49:27.041108 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.041074    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-dda9f5\"/\"openshift-service-ca.crt\""
Oct 30 21:49:27.041108 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.041082    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-dda9f5\"/\"premium-w0-dda9f5-kserve-self-signed-certs\""
Oct 30 21:49:27.041414 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.041328    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-dda9f5\"/\"kube-root-ca.crt\""
Oct 30 21:49:27.119810 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.119778    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-model-cache\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.119810 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.119815    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-tls-certs\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.120074 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.119857    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wmzk\" (UniqueName: \"kubernetes.io/projected/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-kube-api-access-8wmzk\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.120074 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.119900    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-dshm\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.120074 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.119985    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-home\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.120074 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.120005    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-tmp-dir\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.120370 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.120340    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-model-cache\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.120489 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.120379    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-tmp-dir\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.120489 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.120432    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-home\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.122262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.122204    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-dshm\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.122603 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.122584    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-tls-certs\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.153582 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.153549    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:27.180494 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.180462    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wmzk\" (UniqueName: \"kubernetes.io/projected/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-kube-api-access-8wmzk\") pod \"premium-w0-dda9f5-kserve-6b4f99d667-7fnpp\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") " pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.252141 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.252055    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:27.638588 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.638567    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"]
Oct 30 21:49:27.640901 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:49:27.640872    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0546e6b9_2621_4176_a47f_1bdd9e4c4ceb.slice/crio-9f91bd1514c3c6f1ce8544572b6da4e8409d3f0eda40bbb6f11e593a822f342a WatchSource:0}: Error finding container 9f91bd1514c3c6f1ce8544572b6da4e8409d3f0eda40bbb6f11e593a822f342a: Status 404 returned error can't find the container with id 9f91bd1514c3c6f1ce8544572b6da4e8409d3f0eda40bbb6f11e593a822f342a
Oct 30 21:49:27.695395 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:27.695368    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" event={"ID":"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb","Type":"ContainerStarted","Data":"9f91bd1514c3c6f1ce8544572b6da4e8409d3f0eda40bbb6f11e593a822f342a"}
Oct 30 21:49:28.703317 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:28.703226    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" event={"ID":"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb","Type":"ContainerStarted","Data":"c674553328b2747bd7248c0f6085596197d31aa0a5bd6d52999b137b0e3114ea"}
Oct 30 21:49:28.820870 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:28.820815    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podStartSLOduration=2.8207990389999997 podStartE2EDuration="2.820799039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:28.818879413 +0000 UTC m=+911.490783349" watchObservedRunningTime="2026-10-30 21:49:28.820799039 +0000 UTC m=+911.492702973"
Oct 30 21:49:29.252634 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:29.252592    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:31.929836 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:31.929795    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:32.886808 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:32.886771    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"]
Oct 30 21:49:32.890983 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:32.890960    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"
Oct 30 21:49:32.948545 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:32.948508    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"]
Oct 30 21:49:32.988786 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:32.988743    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b857e984-lzbv6\" (UID: \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"
Oct 30 21:49:32.988956 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:32.988888    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ncv9\" (UniqueName: \"kubernetes.io/projected/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-kube-api-access-5ncv9\") pod \"maas-api-revoke-sub-b857e984-lzbv6\" (UID: \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"
Oct 30 21:49:32.989032 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:32.989012    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b857e984-lzbv6\" (UID: \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"
Oct 30 21:49:32.993328 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:32.993277    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:49:32.993453 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:32.993389    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 21:49:33.090191 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:33.090157    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b857e984-lzbv6\" (UID: \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"
Oct 30 21:49:33.090387 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:33.090267    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ncv9\" (UniqueName: \"kubernetes.io/projected/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-kube-api-access-5ncv9\") pod \"maas-api-revoke-sub-b857e984-lzbv6\" (UID: \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"
Oct 30 21:49:33.090387 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:33.090349    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b857e984-lzbv6\" (UID: \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"
Oct 30 21:49:33.090844 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:33.090824    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b857e984-lzbv6\" (UID: \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"
Oct 30 21:49:33.092811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:33.092793    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b857e984-lzbv6\" (UID: \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"
Oct 30 21:49:33.147679 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:33.147606    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ncv9\" (UniqueName: \"kubernetes.io/projected/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-kube-api-access-5ncv9\") pod \"maas-api-revoke-sub-b857e984-lzbv6\" (UID: \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"
Oct 30 21:49:33.206804 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:33.206772    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"
Oct 30 21:49:33.587069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:33.587036    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"]
Oct 30 21:49:33.588809 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:49:33.588776    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9e6bc7d_8a41_43e7_aa99_efcebee0f2a1.slice/crio-ad29acceb34b213125d87fe13e708efcaa187fb846440e3d15c9a76e2290e74d WatchSource:0}: Error finding container ad29acceb34b213125d87fe13e708efcaa187fb846440e3d15c9a76e2290e74d: Status 404 returned error can't find the container with id ad29acceb34b213125d87fe13e708efcaa187fb846440e3d15c9a76e2290e74d
Oct 30 21:49:33.737652 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:33.737608    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6" event={"ID":"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1","Type":"ContainerStarted","Data":"ad29acceb34b213125d87fe13e708efcaa187fb846440e3d15c9a76e2290e74d"}
Oct 30 21:49:34.747810 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:34.747716    5586 generic.go:356] "Generic (PLEG): container finished" podID="c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1" containerID="1f4abcb1cd2ecfb2855a12e74c4ef14ea183c67d308f725820097a8b9239924d" exitCode=0
Oct 30 21:49:34.748184 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:34.747809    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6" event={"ID":"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1","Type":"ContainerDied","Data":"1f4abcb1cd2ecfb2855a12e74c4ef14ea183c67d308f725820097a8b9239924d"}
Oct 30 21:49:35.910059 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:35.910033    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"
Oct 30 21:49:36.022220 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:36.022143    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-maas-api-openshift-ca\") pod \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\" (UID: \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\") "
Oct 30 21:49:36.022220 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:36.022181    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-maas-api-xks-ca\") pod \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\" (UID: \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\") "
Oct 30 21:49:36.022220 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:36.022218    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-kube-api-access-5ncv9\" (UniqueName: \"kubernetes.io/projected/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-kube-api-access-5ncv9\") pod \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\" (UID: \"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1\") "
Oct 30 21:49:36.022569 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:36.022538    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-maas-api-openshift-ca" pod "c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1" (UID: "c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:36.022750 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:36.022734    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:36.024422 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:36.024396    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-maas-api-xks-ca" pod "c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1" (UID: "c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:36.024523 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:36.024428    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-kube-api-access-5ncv9" pod "c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1" (UID: "c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1"). InnerVolumeSpecName "kube-api-access-5ncv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:36.124100 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:36.124061    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:36.124100 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:36.124100    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ncv9\" (UniqueName: \"kubernetes.io/projected/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1-kube-api-access-5ncv9\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:36.707740 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:36.707698    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:36.761890 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:36.761851    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6" event={"ID":"c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1","Type":"ContainerDied","Data":"ad29acceb34b213125d87fe13e708efcaa187fb846440e3d15c9a76e2290e74d"}
Oct 30 21:49:36.761890 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:36.761873    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"
Oct 30 21:49:36.762090 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:36.761899    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad29acceb34b213125d87fe13e708efcaa187fb846440e3d15c9a76e2290e74d"
Oct 30 21:49:37.253124 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:37.253092    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:37.265918 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:37.265888    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:37.779081 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:37.779042    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:49:40.858989 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:40.858952    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:41.431672 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:41.431632    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:46.196700 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:46.196656    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:48.362300 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:49:48.362220    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:49:50.976674 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:50.976630    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:55.781992 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:55.781955    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:57.297987 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.297952    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"]
Oct 30 21:49:57.298539 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.298525    5586 cpu_manager.go:414] "containerMap: removing container" podUID="c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1" containerName="revoke-keys"
Oct 30 21:49:57.298584 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.298540    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1" containerName="revoke-keys"
Oct 30 21:49:57.298657 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.298647    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1" containerName="revoke-keys"
Oct 30 21:49:57.302378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.302360    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.344071 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.344030    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:57.354548 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.354516    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-80c366-openshift-default-dockercfg-jqtkq\""
Oct 30 21:49:57.390753 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.390714    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.391160 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.391134    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.391407 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.391385    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.391622 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.391605    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6hct\" (UniqueName: \"kubernetes.io/projected/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-kube-api-access-f6hct\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.391857 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.391765    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.391982 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.391899    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.392047 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.391987    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.392047 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.392017    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.392159 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.392137    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.410188 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.410157    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"]
Oct 30 21:49:57.493136 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.493098    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.493345 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.493178    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.493345 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.493205    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.493345 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.493280    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.493345 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.493326    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.493582 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.493373    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.493582 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.493421    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.493582 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.493505    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6hct\" (UniqueName: \"kubernetes.io/projected/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-kube-api-access-f6hct\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.493582 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.493540    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.493786 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.493718    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.493893 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.493839    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.493953 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.493875    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.494154 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.494125    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.494468 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.494445    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.495625 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.495604    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.496119 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.496095    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.501584 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.501554    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:57.554041 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.553956    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6hct\" (UniqueName: \"kubernetes.io/projected/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-kube-api-access-f6hct\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.554041 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.553979    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:57.616908 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:57.616872    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:58.051850 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.051821    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"]
Oct 30 21:49:58.053453 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:49:58.053422    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd67ce59f_5a5b_449e_9fcb_1207b9e151d3.slice/crio-948964a6ce0e17c1941c4737821ea5d633c74e68fda2f5716c731dcf1c151a9b WatchSource:0}: Error finding container 948964a6ce0e17c1941c4737821ea5d633c74e68fda2f5716c731dcf1c151a9b: Status 404 returned error can't find the container with id 948964a6ce0e17c1941c4737821ea5d633c74e68fda2f5716c731dcf1c151a9b
Oct 30 21:49:58.055709 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.055679    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:58.055769 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.055747    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:58.055805 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.055777    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:58.284530 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.284490    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"]
Oct 30 21:49:58.289090 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.289062    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.343927 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.343222    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-dda9f5-openshift-default-dockercfg-qx2fq\""
Oct 30 21:49:58.347144 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.347108    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"]
Oct 30 21:49:58.404278 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.404227    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.404503 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.404288    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.404503 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.404416    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w46dq\" (UniqueName: \"kubernetes.io/projected/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-kube-api-access-w46dq\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.404503 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.404468    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.404671 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.404547    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.404671 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.404579    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.404751 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.404669    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.404751 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.404700    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.404840 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.404776    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.505850 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.505802    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w46dq\" (UniqueName: \"kubernetes.io/projected/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-kube-api-access-w46dq\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.506017 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.505859    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.506017 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.505905    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.506017 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.505943    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.506218 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.506084    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.506218 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.506133    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.506218 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.506167    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.506388 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.506375    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.506450 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.506421    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.506500 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.506456    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.506571 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.506548    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.506632 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.506549    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.506791 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.506765    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.507084 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.507062    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.508395 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.508373    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.508939 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.508913    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.567301 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.567259    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.567481 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.567458    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w46dq\" (UniqueName: \"kubernetes.io/projected/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-kube-api-access-w46dq\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.606360 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.606273    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:49:58.948763 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.948720    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2" event={"ID":"d67ce59f-5a5b-449e-9fcb-1207b9e151d3","Type":"ContainerStarted","Data":"dff88d61eb6c4e318bd514ec21b73885922aba1339f472dcaa8cc98dfa74969c"}
Oct 30 21:49:58.948950 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:58.948789    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2" event={"ID":"d67ce59f-5a5b-449e-9fcb-1207b9e151d3","Type":"ContainerStarted","Data":"948964a6ce0e17c1941c4737821ea5d633c74e68fda2f5716c731dcf1c151a9b"}
Oct 30 21:49:59.052749 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:59.052717    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"]
Oct 30 21:49:59.053611 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:49:59.053582    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbe524d9_e536_4990_ac0b_8ad75c9fd52e.slice/crio-9201fc5ae1d8cc818ad92af5af0f76191c924d2a43ae64d05dc5c82e90867f73 WatchSource:0}: Error finding container 9201fc5ae1d8cc818ad92af5af0f76191c924d2a43ae64d05dc5c82e90867f73: Status 404 returned error can't find the container with id 9201fc5ae1d8cc818ad92af5af0f76191c924d2a43ae64d05dc5c82e90867f73
Oct 30 21:49:59.062773 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:59.056774    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:59.062773 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:59.056935    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:59.062773 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:59.057021    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:49:59.068723 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:59.068670    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2" podStartSLOduration=2.068652858 podStartE2EDuration="2.068652858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:59.067213052 +0000 UTC m=+941.739116990" watchObservedRunningTime="2026-10-30 21:49:59.068652858 +0000 UTC m=+941.740556794"
Oct 30 21:49:59.619346 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:59.619181    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:49:59.630266 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:59.624943    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 21:49:59.630266 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:59.625011    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2" podUID="d67ce59f-5a5b-449e-9fcb-1207b9e151d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 21:49:59.975286 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:59.974456    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv" event={"ID":"bbe524d9-e536-4990-ac0b-8ad75c9fd52e","Type":"ContainerStarted","Data":"667f837adaf4fe3b18d5f6d7ae00dd52971dfaf8de5cf2be9f354f3f7e40eaef"}
Oct 30 21:49:59.975286 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:49:59.974523    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv" event={"ID":"bbe524d9-e536-4990-ac0b-8ad75c9fd52e","Type":"ContainerStarted","Data":"9201fc5ae1d8cc818ad92af5af0f76191c924d2a43ae64d05dc5c82e90867f73"}
Oct 30 21:50:00.100604 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:00.100543    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv" podStartSLOduration=2.10052898 podStartE2EDuration="2.10052898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:00.099676311 +0000 UTC m=+942.771580248" watchObservedRunningTime="2026-10-30 21:50:00.10052898 +0000 UTC m=+942.772432914"
Oct 30 21:50:00.599436 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:00.599398    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:00.607299 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:00.607266    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:50:00.608425 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:00.608383    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:50:00.608575 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:00.608446    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv" podUID="bbe524d9-e536-4990-ac0b-8ad75c9fd52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 21:50:00.617955 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:00.617901    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 21:50:00.618133 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:00.617988    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2" podUID="d67ce59f-5a5b-449e-9fcb-1207b9e151d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 21:50:01.607183 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:01.607146    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:50:01.607628 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:01.607201    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv" podUID="bbe524d9-e536-4990-ac0b-8ad75c9fd52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 21:50:01.617630 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:01.617605    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 21:50:01.617704 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:01.617645    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2" podUID="d67ce59f-5a5b-449e-9fcb-1207b9e151d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 21:50:02.606940 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:02.606900    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:50:02.607127 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:02.606957    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv" podUID="bbe524d9-e536-4990-ac0b-8ad75c9fd52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 21:50:02.617286 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:02.617258    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 21:50:02.617635 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:02.617314    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2" podUID="d67ce59f-5a5b-449e-9fcb-1207b9e151d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 21:50:03.606840 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:03.606804    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:50:03.607038 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:03.606866    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv" podUID="bbe524d9-e536-4990-ac0b-8ad75c9fd52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 21:50:03.617175 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:03.617147    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 21:50:03.617391 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:03.617369    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2" podUID="d67ce59f-5a5b-449e-9fcb-1207b9e151d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 21:50:04.606807 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:04.606765    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:50:04.607064 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:04.606831    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv" podUID="bbe524d9-e536-4990-ac0b-8ad75c9fd52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 21:50:04.617158 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:04.617124    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 21:50:04.617300 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:04.617167    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2" podUID="d67ce59f-5a5b-449e-9fcb-1207b9e151d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 21:50:05.396442 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:05.396404    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:05.607588 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:05.607554    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:50:05.607759 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:05.607606    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv" podUID="bbe524d9-e536-4990-ac0b-8ad75c9fd52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 21:50:05.622351 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:05.622323    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:50:05.622663 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:05.622637    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:50:05.623471 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:05.623452    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 21:50:06.611775 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:06.611743    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:50:06.612200 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:06.612151    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:50:06.612786 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:06.612765    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 21:50:10.282607 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:10.282572    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:13.206964 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:13.206929    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:15.066374 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:15.066324    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:18.363205 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:50:18.363165    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:50:19.918301 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:19.918035    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:20.558561 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:20.558505    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:24.687998 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:24.687961    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:29.495789 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:29.495742    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:34.314198 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:34.314153    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:39.080173 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:39.080136    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:43.876722 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:43.876685    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:48.361175 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:50:48.361141    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:50:48.678753 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:48.678721    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:53.477193 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:53.477155    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:54.537602 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:54.537559    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:56.808882 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:56.808852    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"]
Oct 30 21:50:56.813157 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:56.813139    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:56.917444 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:56.917409    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"]
Oct 30 21:50:56.926931 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:56.926897    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:56.927125 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:56.926956    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:56.927196 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:56.927129    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:56.927326 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:56.927226    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:56.927433 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:56.927410    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:56.927498 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:56.927482    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27b11c10-67cb-4786-8f9a-70afaa107f77-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:56.927555 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:56.927503    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:56.927555 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:56.927549    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:56.927668 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:56.927579    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqmgj\" (UniqueName: \"kubernetes.io/projected/27b11c10-67cb-4786-8f9a-70afaa107f77-kube-api-access-vqmgj\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.028664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.028621    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.028864 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.028674    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.028864 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.028789    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.028988 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.028891    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.028988 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.028950    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27b11c10-67cb-4786-8f9a-70afaa107f77-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.028988 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.028978    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.029143 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.029032    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.029143 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.029068    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.029143 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.029079    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqmgj\" (UniqueName: \"kubernetes.io/projected/27b11c10-67cb-4786-8f9a-70afaa107f77-kube-api-access-vqmgj\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.029307 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.029161    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.029871 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.029802    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.029871 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.029845    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.030054 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.029974    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.030118 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.030067    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27b11c10-67cb-4786-8f9a-70afaa107f77-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.031446 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.031416    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.031553 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.031501    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.086826 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.086749    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.086966 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.086849    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqmgj\" (UniqueName: \"kubernetes.io/projected/27b11c10-67cb-4786-8f9a-70afaa107f77-kube-api-access-vqmgj\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.126755 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.126725    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:57.557389 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.557357    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"]
Oct 30 21:50:57.558552 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:50:57.558522    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27b11c10_67cb_4786_8f9a_70afaa107f77.slice/crio-9493319118c19dd345941c34ebb54269ea974c32a1257d4b8336991416c97cb8 WatchSource:0}: Error finding container 9493319118c19dd345941c34ebb54269ea974c32a1257d4b8336991416c97cb8: Status 404 returned error can't find the container with id 9493319118c19dd345941c34ebb54269ea974c32a1257d4b8336991416c97cb8
Oct 30 21:50:57.561001 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.560969    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:50:57.561092 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.561043    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:50:57.561092 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:57.561079    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:50:58.242668 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:58.241472    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:58.487884 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:58.487790    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj" event={"ID":"27b11c10-67cb-4786-8f9a-70afaa107f77","Type":"ContainerStarted","Data":"3a1adcbd62e9cf082fad2d6f7dabdfe7cac37ad5cd13b75cdf522d6fd9142d76"}
Oct 30 21:50:58.488250 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:58.488203    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj" event={"ID":"27b11c10-67cb-4786-8f9a-70afaa107f77","Type":"ContainerStarted","Data":"9493319118c19dd345941c34ebb54269ea974c32a1257d4b8336991416c97cb8"}
Oct 30 21:50:58.749255 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:58.748799    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj" podStartSLOduration=2.7487763469999997 podStartE2EDuration="2.748776347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:58.693954694 +0000 UTC m=+1001.365858629" watchObservedRunningTime="2026-10-30 21:50:58.748776347 +0000 UTC m=+1001.420680283"
Oct 30 21:50:59.127769 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:59.127632    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:50:59.130991 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:59.130595    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:50:59.130991 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:50:59.130663    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj" podUID="27b11c10-67cb-4786-8f9a-70afaa107f77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:51:00.136259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:00.131736    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:51:00.136259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:00.131815    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj" podUID="27b11c10-67cb-4786-8f9a-70afaa107f77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:51:01.134682 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:01.134640    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:51:01.134981 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:01.134954    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj" podUID="27b11c10-67cb-4786-8f9a-70afaa107f77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:51:02.148141 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:02.148088    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:51:02.148740 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:02.148705    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj" podUID="27b11c10-67cb-4786-8f9a-70afaa107f77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:51:03.130718 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:03.130665    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:51:03.131050 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:03.131005    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj" podUID="27b11c10-67cb-4786-8f9a-70afaa107f77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:51:03.344258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:03.343963    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:04.127701 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:04.127665    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:51:04.127899 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:04.127729    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj" podUID="27b11c10-67cb-4786-8f9a-70afaa107f77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:51:05.363666 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:05.363621    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:51:05.364417 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:05.364393    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:51:05.365194 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:05.365173    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 21:51:08.151854 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:08.151819    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:12.351983 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:12.351939    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:12.977801 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:12.977766    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:14.519674 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:14.519633    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"]
Oct 30 21:51:14.520082 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:14.519842    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4" podUID="6930c8f5-9706-4db9-8957-8f209fc889a0" containerName="istio-proxy" containerID="cri-o://0043a83249f88e42fb2bc8892018dc434184f27ed438a98449e85604b405f668" gracePeriod=30
Oct 30 21:51:15.448569 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.448536    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"]
Oct 30 21:51:15.452619 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.452595    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"
Oct 30 21:51:15.508928 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.508896    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"]
Oct 30 21:51:15.511026 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.510981    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/50340bb1-68cf-4f15-bd32-5ad69697188c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-784d86fa-pzpsc\" (UID: \"50340bb1-68cf-4f15-bd32-5ad69697188c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"
Oct 30 21:51:15.511216 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.511114    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qng8c\" (UniqueName: \"kubernetes.io/projected/50340bb1-68cf-4f15-bd32-5ad69697188c-kube-api-access-qng8c\") pod \"maas-api-revoke-sub-784d86fa-pzpsc\" (UID: \"50340bb1-68cf-4f15-bd32-5ad69697188c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"
Oct 30 21:51:15.511435 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.511372    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/50340bb1-68cf-4f15-bd32-5ad69697188c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-784d86fa-pzpsc\" (UID: \"50340bb1-68cf-4f15-bd32-5ad69697188c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"
Oct 30 21:51:15.553454 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.553426    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:51:15.553454 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.553440    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 21:51:15.612097 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.612061    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/50340bb1-68cf-4f15-bd32-5ad69697188c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-784d86fa-pzpsc\" (UID: \"50340bb1-68cf-4f15-bd32-5ad69697188c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"
Oct 30 21:51:15.612305 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.612183    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/50340bb1-68cf-4f15-bd32-5ad69697188c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-784d86fa-pzpsc\" (UID: \"50340bb1-68cf-4f15-bd32-5ad69697188c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"
Oct 30 21:51:15.612356 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.612305    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qng8c\" (UniqueName: \"kubernetes.io/projected/50340bb1-68cf-4f15-bd32-5ad69697188c-kube-api-access-qng8c\") pod \"maas-api-revoke-sub-784d86fa-pzpsc\" (UID: \"50340bb1-68cf-4f15-bd32-5ad69697188c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"
Oct 30 21:51:15.612857 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.612828    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/50340bb1-68cf-4f15-bd32-5ad69697188c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-784d86fa-pzpsc\" (UID: \"50340bb1-68cf-4f15-bd32-5ad69697188c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"
Oct 30 21:51:15.614782 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.614755    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/50340bb1-68cf-4f15-bd32-5ad69697188c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-784d86fa-pzpsc\" (UID: \"50340bb1-68cf-4f15-bd32-5ad69697188c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"
Oct 30 21:51:15.669636 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.669604    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qng8c\" (UniqueName: \"kubernetes.io/projected/50340bb1-68cf-4f15-bd32-5ad69697188c-kube-api-access-qng8c\") pod \"maas-api-revoke-sub-784d86fa-pzpsc\" (UID: \"50340bb1-68cf-4f15-bd32-5ad69697188c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"
Oct 30 21:51:15.767220 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:15.767140    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"
Oct 30 21:51:16.144902 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:16.144875    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"]
Oct 30 21:51:16.146471 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:51:16.146445    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50340bb1_68cf_4f15_bd32_5ad69697188c.slice/crio-882ccb8edf7349441d1949c87b33399bd6472d7ef65c6d6c8e0f6d289648c493 WatchSource:0}: Error finding container 882ccb8edf7349441d1949c87b33399bd6472d7ef65c6d6c8e0f6d289648c493: Status 404 returned error can't find the container with id 882ccb8edf7349441d1949c87b33399bd6472d7ef65c6d6c8e0f6d289648c493
Oct 30 21:51:16.607592 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:16.607553    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc" event={"ID":"50340bb1-68cf-4f15-bd32-5ad69697188c","Type":"ContainerStarted","Data":"c383588f6978ab54ec08ffbec2a66f7fe53f5d28d749cc65f7c9135e401c95a4"}
Oct 30 21:51:16.607999 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:16.607613    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc" event={"ID":"50340bb1-68cf-4f15-bd32-5ad69697188c","Type":"ContainerStarted","Data":"882ccb8edf7349441d1949c87b33399bd6472d7ef65c6d6c8e0f6d289648c493"}
Oct 30 21:51:16.726269 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:16.726156    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc" podStartSLOduration=1.726142152 podStartE2EDuration="1.726142152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:16.723559503 +0000 UTC m=+1019.395463440" watchObservedRunningTime="2026-10-30 21:51:16.726142152 +0000 UTC m=+1019.398046087"
Oct 30 21:51:17.615707 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:17.615670    5586 generic.go:356] "Generic (PLEG): container finished" podID="50340bb1-68cf-4f15-bd32-5ad69697188c" containerID="c383588f6978ab54ec08ffbec2a66f7fe53f5d28d749cc65f7c9135e401c95a4" exitCode=0
Oct 30 21:51:17.616077 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:17.615733    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc" event={"ID":"50340bb1-68cf-4f15-bd32-5ad69697188c","Type":"ContainerDied","Data":"c383588f6978ab54ec08ffbec2a66f7fe53f5d28d749cc65f7c9135e401c95a4"}
Oct 30 21:51:17.716920 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:17.716885    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:18.362305 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:51:18.362265    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:51:18.761609 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:18.761578    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"
Oct 30 21:51:18.850096 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:18.850057    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/50340bb1-68cf-4f15-bd32-5ad69697188c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/50340bb1-68cf-4f15-bd32-5ad69697188c-maas-api-xks-ca\") pod \"50340bb1-68cf-4f15-bd32-5ad69697188c\" (UID: \"50340bb1-68cf-4f15-bd32-5ad69697188c\") "
Oct 30 21:51:18.850322 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:18.850192    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50340bb1-68cf-4f15-bd32-5ad69697188c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/50340bb1-68cf-4f15-bd32-5ad69697188c-maas-api-openshift-ca\") pod \"50340bb1-68cf-4f15-bd32-5ad69697188c\" (UID: \"50340bb1-68cf-4f15-bd32-5ad69697188c\") "
Oct 30 21:51:18.850322 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:18.850221    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50340bb1-68cf-4f15-bd32-5ad69697188c-kube-api-access-qng8c\" (UniqueName: \"kubernetes.io/projected/50340bb1-68cf-4f15-bd32-5ad69697188c-kube-api-access-qng8c\") pod \"50340bb1-68cf-4f15-bd32-5ad69697188c\" (UID: \"50340bb1-68cf-4f15-bd32-5ad69697188c\") "
Oct 30 21:51:18.850661 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:18.850632    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50340bb1-68cf-4f15-bd32-5ad69697188c-maas-api-openshift-ca" pod "50340bb1-68cf-4f15-bd32-5ad69697188c" (UID: "50340bb1-68cf-4f15-bd32-5ad69697188c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:18.852380 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:18.852349    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50340bb1-68cf-4f15-bd32-5ad69697188c-maas-api-xks-ca" pod "50340bb1-68cf-4f15-bd32-5ad69697188c" (UID: "50340bb1-68cf-4f15-bd32-5ad69697188c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:18.852500 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:18.852415    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50340bb1-68cf-4f15-bd32-5ad69697188c-kube-api-access-qng8c" pod "50340bb1-68cf-4f15-bd32-5ad69697188c" (UID: "50340bb1-68cf-4f15-bd32-5ad69697188c"). InnerVolumeSpecName "kube-api-access-qng8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:18.951810 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:18.951763    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/50340bb1-68cf-4f15-bd32-5ad69697188c-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:18.951810 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:18.951803    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/50340bb1-68cf-4f15-bd32-5ad69697188c-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:18.951810 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:18.951817    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qng8c\" (UniqueName: \"kubernetes.io/projected/50340bb1-68cf-4f15-bd32-5ad69697188c-kube-api-access-qng8c\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:19.648375 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.646321    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc" event={"ID":"50340bb1-68cf-4f15-bd32-5ad69697188c","Type":"ContainerDied","Data":"882ccb8edf7349441d1949c87b33399bd6472d7ef65c6d6c8e0f6d289648c493"}
Oct 30 21:51:19.648375 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.646420    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="882ccb8edf7349441d1949c87b33399bd6472d7ef65c6d6c8e0f6d289648c493"
Oct 30 21:51:19.648375 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.646492    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"
Oct 30 21:51:19.941203 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.940865    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:51:19.967542 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.967504    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-data\") pod \"6930c8f5-9706-4db9-8957-8f209fc889a0\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") "
Oct 30 21:51:19.967764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.967555    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6930c8f5-9706-4db9-8957-8f209fc889a0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6930c8f5-9706-4db9-8957-8f209fc889a0-istiod-ca-cert\") pod \"6930c8f5-9706-4db9-8957-8f209fc889a0\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") "
Oct 30 21:51:19.967764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.967703    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-workload-socket\") pod \"6930c8f5-9706-4db9-8957-8f209fc889a0\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") "
Oct 30 21:51:19.967764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.967737    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-podinfo\") pod \"6930c8f5-9706-4db9-8957-8f209fc889a0\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") "
Oct 30 21:51:19.967959 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.967794    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6930c8f5-9706-4db9-8957-8f209fc889a0-kube-api-access-kxnmk\" (UniqueName: \"kubernetes.io/projected/6930c8f5-9706-4db9-8957-8f209fc889a0-kube-api-access-kxnmk\") pod \"6930c8f5-9706-4db9-8957-8f209fc889a0\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") "
Oct 30 21:51:19.967959 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.967815    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-data" pod "6930c8f5-9706-4db9-8957-8f209fc889a0" (UID: "6930c8f5-9706-4db9-8957-8f209fc889a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:19.967959 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.967828    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-envoy\") pod \"6930c8f5-9706-4db9-8957-8f209fc889a0\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") "
Oct 30 21:51:19.967959 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.967878    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-credential-socket\") pod \"6930c8f5-9706-4db9-8957-8f209fc889a0\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") "
Oct 30 21:51:19.967959 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.967909    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-workload-certs\") pod \"6930c8f5-9706-4db9-8957-8f209fc889a0\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") "
Oct 30 21:51:19.968267 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.967969    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-token\" (UniqueName: \"kubernetes.io/projected/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-token\") pod \"6930c8f5-9706-4db9-8957-8f209fc889a0\" (UID: \"6930c8f5-9706-4db9-8957-8f209fc889a0\") "
Oct 30 21:51:19.968267 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.968045    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6930c8f5-9706-4db9-8957-8f209fc889a0-istiod-ca-cert" pod "6930c8f5-9706-4db9-8957-8f209fc889a0" (UID: "6930c8f5-9706-4db9-8957-8f209fc889a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:19.968885 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.968397    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-workload-socket" pod "6930c8f5-9706-4db9-8957-8f209fc889a0" (UID: "6930c8f5-9706-4db9-8957-8f209fc889a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:19.968885 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.968559    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:19.968885 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.968579    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6930c8f5-9706-4db9-8957-8f209fc889a0-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:19.968885 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.968594    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:19.969567 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.969507    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-workload-certs" pod "6930c8f5-9706-4db9-8957-8f209fc889a0" (UID: "6930c8f5-9706-4db9-8957-8f209fc889a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:19.969884 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.969864    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-credential-socket" pod "6930c8f5-9706-4db9-8957-8f209fc889a0" (UID: "6930c8f5-9706-4db9-8957-8f209fc889a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:19.970972 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.970943    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-envoy" pod "6930c8f5-9706-4db9-8957-8f209fc889a0" (UID: "6930c8f5-9706-4db9-8957-8f209fc889a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:19.971082 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.971061    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-token" pod "6930c8f5-9706-4db9-8957-8f209fc889a0" (UID: "6930c8f5-9706-4db9-8957-8f209fc889a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:19.971609 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.971575    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6930c8f5-9706-4db9-8957-8f209fc889a0-kube-api-access-kxnmk" pod "6930c8f5-9706-4db9-8957-8f209fc889a0" (UID: "6930c8f5-9706-4db9-8957-8f209fc889a0"). InnerVolumeSpecName "kube-api-access-kxnmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:19.972357 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:19.972331    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-podinfo" pod "6930c8f5-9706-4db9-8957-8f209fc889a0" (UID: "6930c8f5-9706-4db9-8957-8f209fc889a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:20.076366 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.071116    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:20.076366 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.071160    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:20.076366 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.071179    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6930c8f5-9706-4db9-8957-8f209fc889a0-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:20.076366 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.071193    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:20.076366 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.071207    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6930c8f5-9706-4db9-8957-8f209fc889a0-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:20.076821 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.071222    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxnmk\" (UniqueName: \"kubernetes.io/projected/6930c8f5-9706-4db9-8957-8f209fc889a0-kube-api-access-kxnmk\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:51:20.704258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.703980    5586 generic.go:356] "Generic (PLEG): container finished" podID="6930c8f5-9706-4db9-8957-8f209fc889a0" containerID="0043a83249f88e42fb2bc8892018dc434184f27ed438a98449e85604b405f668" exitCode=0
Oct 30 21:51:20.704258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.704085    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4" event={"ID":"6930c8f5-9706-4db9-8957-8f209fc889a0","Type":"ContainerDied","Data":"0043a83249f88e42fb2bc8892018dc434184f27ed438a98449e85604b405f668"}
Oct 30 21:51:20.704258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.704138    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4" event={"ID":"6930c8f5-9706-4db9-8957-8f209fc889a0","Type":"ContainerDied","Data":"d032f95078e9d8d9d8a965f10f2e158e9b78594e34c0ddce649de3346674d7fa"}
Oct 30 21:51:20.704258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.704169    5586 scope.go:122] "RemoveContainer" containerID="0043a83249f88e42fb2bc8892018dc434184f27ed438a98449e85604b405f668"
Oct 30 21:51:20.704601 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.704359    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"
Oct 30 21:51:20.828183 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.828152    5586 scope.go:122] "RemoveContainer" containerID="0043a83249f88e42fb2bc8892018dc434184f27ed438a98449e85604b405f668"
Oct 30 21:51:20.828873 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:51:20.828832    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0043a83249f88e42fb2bc8892018dc434184f27ed438a98449e85604b405f668\": container with ID starting with 0043a83249f88e42fb2bc8892018dc434184f27ed438a98449e85604b405f668 not found: ID does not exist" containerID="0043a83249f88e42fb2bc8892018dc434184f27ed438a98449e85604b405f668"
Oct 30 21:51:20.829091 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.829043    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0043a83249f88e42fb2bc8892018dc434184f27ed438a98449e85604b405f668"} err="failed to get container status \"0043a83249f88e42fb2bc8892018dc434184f27ed438a98449e85604b405f668\": rpc error: code = NotFound desc = could not find container \"0043a83249f88e42fb2bc8892018dc434184f27ed438a98449e85604b405f668\": container with ID starting with 0043a83249f88e42fb2bc8892018dc434184f27ed438a98449e85604b405f668 not found: ID does not exist"
Oct 30 21:51:20.916806 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.916762    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:20.950268 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.949916    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"]
Oct 30 21:51:20.995575 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:20.995484    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mglq4"]
Oct 30 21:51:21.447460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:21.447429    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6930c8f5-9706-4db9-8957-8f209fc889a0" path="/var/lib/kubelet/pods/6930c8f5-9706-4db9-8957-8f209fc889a0/volumes"
Oct 30 21:51:22.802670 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:22.802635    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:27.684872 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:27.684821    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:32.505661 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:32.505619    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:37.275394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:37.275351    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:42.186927 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.186892    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:42.783254 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.782974    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"]
Oct 30 21:51:42.789351 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.783802    5586 cpu_manager.go:414] "containerMap: removing container" podUID="6930c8f5-9706-4db9-8957-8f209fc889a0" containerName="istio-proxy"
Oct 30 21:51:42.789351 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.783826    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6930c8f5-9706-4db9-8957-8f209fc889a0" containerName="istio-proxy"
Oct 30 21:51:42.789351 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.783846    5586 cpu_manager.go:414] "containerMap: removing container" podUID="50340bb1-68cf-4f15-bd32-5ad69697188c" containerName="revoke-keys"
Oct 30 21:51:42.789351 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.783856    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50340bb1-68cf-4f15-bd32-5ad69697188c" containerName="revoke-keys"
Oct 30 21:51:42.789351 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.783985    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="6930c8f5-9706-4db9-8957-8f209fc889a0" containerName="istio-proxy"
Oct 30 21:51:42.789351 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.783997    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="50340bb1-68cf-4f15-bd32-5ad69697188c" containerName="revoke-keys"
Oct 30 21:51:42.789351 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.787539    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:42.846548 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.846519    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"]
Oct 30 21:51:42.891450 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.891422    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-c79cb5-openshift-default-dockercfg-7289d\""
Oct 30 21:51:42.969487 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.969445    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:42.969685 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.969499    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:42.969685 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.969589    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:42.969685 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.969652    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5989307-0754-4796-a7f4-79a20accfbd5-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:42.969852 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.969690    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bq68\" (UniqueName: \"kubernetes.io/projected/a5989307-0754-4796-a7f4-79a20accfbd5-kube-api-access-7bq68\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:42.969852 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.969780    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:42.969852 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.969809    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5989307-0754-4796-a7f4-79a20accfbd5-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:42.969974 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.969957    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5989307-0754-4796-a7f4-79a20accfbd5-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:42.970009 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:42.969986    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.071142 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.071036    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5989307-0754-4796-a7f4-79a20accfbd5-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.071142 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.071081    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.071142 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.071134    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.071461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.071160    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.071461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.071177    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.071461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.071211    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5989307-0754-4796-a7f4-79a20accfbd5-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.071461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.071271    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bq68\" (UniqueName: \"kubernetes.io/projected/a5989307-0754-4796-a7f4-79a20accfbd5-kube-api-access-7bq68\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.071666 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.071314    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.071719 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.071669    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5989307-0754-4796-a7f4-79a20accfbd5-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.071719 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.071681    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.071719 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.071699    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.072010 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.071946    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.072214 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.072191    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.072707 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.072685    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5989307-0754-4796-a7f4-79a20accfbd5-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.073752 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.073725    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.073911 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.073889    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5989307-0754-4796-a7f4-79a20accfbd5-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.118145 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.118110    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"]
Oct 30 21:51:43.122557 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.122535    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.129148 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.129117    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5989307-0754-4796-a7f4-79a20accfbd5-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.129286 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.129260    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bq68\" (UniqueName: \"kubernetes.io/projected/a5989307-0754-4796-a7f4-79a20accfbd5-kube-api-access-7bq68\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.181707 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.181670    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"]
Oct 30 21:51:43.273512 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.273479    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.273512 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.273510    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.273976 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.273608    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsssd\" (UniqueName: \"kubernetes.io/projected/e00cca75-dfc2-4231-ad1c-61c7069faa38-kube-api-access-xsssd\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.273976 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.273703    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.273976 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.273741    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.273976 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.273835    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.273976 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.273861    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.273976 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.273902    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00cca75-dfc2-4231-ad1c-61c7069faa38-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.273976 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.273959    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.375425 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.375329    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.375425 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.375380    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.375666 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.375431    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.375666 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.375449    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.375666 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.375502    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00cca75-dfc2-4231-ad1c-61c7069faa38-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.375666 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.375527    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.375666 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.375620    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.375666 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.375644    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.376061 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.375700    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsssd\" (UniqueName: \"kubernetes.io/projected/e00cca75-dfc2-4231-ad1c-61c7069faa38-kube-api-access-xsssd\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.376061 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.375828    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.376061 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.375839    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.376061 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.375995    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.376311 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.376075    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.376518 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.376494    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00cca75-dfc2-4231-ad1c-61c7069faa38-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.377895 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.377872    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.378107 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.378090    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.402978 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.402940    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:43.445924 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.445890    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsssd\" (UniqueName: \"kubernetes.io/projected/e00cca75-dfc2-4231-ad1c-61c7069faa38-kube-api-access-xsssd\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.446869 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.446817    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.449822 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.449796    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:43.738046 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.738011    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"]
Oct 30 21:51:43.743375 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.743347    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.803125 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.803093    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"]
Oct 30 21:51:43.834753 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.834714    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"]
Oct 30 21:51:43.854054 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:51:43.854024    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5989307_0754_4796_a7f4_79a20accfbd5.slice/crio-9f652a611a3902b5ee6d673f34c23f9c49e108a2b3cb1d19db313ce25b35d52c WatchSource:0}: Error finding container 9f652a611a3902b5ee6d673f34c23f9c49e108a2b3cb1d19db313ce25b35d52c: Status 404 returned error can't find the container with id 9f652a611a3902b5ee6d673f34c23f9c49e108a2b3cb1d19db313ce25b35d52c
Oct 30 21:51:43.856478 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.856446    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:43.856563 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.856518    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:43.856563 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.856548    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:43.881330 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.881302    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/146b8994-5857-43e6-a00e-40683229192f-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.881443 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.881421    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.881503 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.881490    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/146b8994-5857-43e6-a00e-40683229192f-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.881542 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.881523    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.881577 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.881555    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk46v\" (UniqueName: \"kubernetes.io/projected/146b8994-5857-43e6-a00e-40683229192f-kube-api-access-nk46v\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.881642 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.881631    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.881675 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.881649    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/146b8994-5857-43e6-a00e-40683229192f-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.881709 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.881688    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.881821 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.881807    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.889667 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.889637    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"]
Oct 30 21:51:43.890765 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:51:43.890742    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00cca75_dfc2_4231_ad1c_61c7069faa38.slice/crio-2b12a3f966b483859004f4baa78482ca55f2a57bfb365c8a00ae51eeb4ef376b WatchSource:0}: Error finding container 2b12a3f966b483859004f4baa78482ca55f2a57bfb365c8a00ae51eeb4ef376b: Status 404 returned error can't find the container with id 2b12a3f966b483859004f4baa78482ca55f2a57bfb365c8a00ae51eeb4ef376b
Oct 30 21:51:43.893283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.893251    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:43.893387 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.893330    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:43.893387 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.893377    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:43.983448 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.983412    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/146b8994-5857-43e6-a00e-40683229192f-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.983650 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.983465    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.983650 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.983497    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk46v\" (UniqueName: \"kubernetes.io/projected/146b8994-5857-43e6-a00e-40683229192f-kube-api-access-nk46v\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.983650 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.983550    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.983650 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.983571    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/146b8994-5857-43e6-a00e-40683229192f-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.983650 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.983616    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.983650 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.983649    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.983996 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.983684    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/146b8994-5857-43e6-a00e-40683229192f-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.983996 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.983756    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.983996 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.983865    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.984153 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.984037    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.984153 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.984062    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.984487 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.984463    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/146b8994-5857-43e6-a00e-40683229192f-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.984719 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.984696    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.986422 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.986392    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/146b8994-5857-43e6-a00e-40683229192f-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:43.986854 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:43.986827    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:44.040005 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:44.039935    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/146b8994-5857-43e6-a00e-40683229192f-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:44.041971 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:44.041947    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk46v\" (UniqueName: \"kubernetes.io/projected/146b8994-5857-43e6-a00e-40683229192f-kube-api-access-nk46v\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:44.058896 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:44.058863    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:44.462411 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:44.462369    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"]
Oct 30 21:51:44.481034 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:51:44.480258    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod146b8994_5857_43e6_a00e_40683229192f.slice/crio-d9664c54caeadc9351e1203e3657846989e98e5a34f4fb698edd4eb779163db9 WatchSource:0}: Error finding container d9664c54caeadc9351e1203e3657846989e98e5a34f4fb698edd4eb779163db9: Status 404 returned error can't find the container with id d9664c54caeadc9351e1203e3657846989e98e5a34f4fb698edd4eb779163db9
Oct 30 21:51:44.483456 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:44.483414    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:44.483563 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:44.483507    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:44.483563 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:44.483551    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:51:44.881309 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:44.880811    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" event={"ID":"146b8994-5857-43e6-a00e-40683229192f","Type":"ContainerStarted","Data":"f2830b4a27ce15d036f400720112f600d17d967c89e685068da4c44e2e1f657a"}
Oct 30 21:51:44.881309 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:44.880878    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" event={"ID":"146b8994-5857-43e6-a00e-40683229192f","Type":"ContainerStarted","Data":"d9664c54caeadc9351e1203e3657846989e98e5a34f4fb698edd4eb779163db9"}
Oct 30 21:51:44.883604 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:44.882651    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5" event={"ID":"a5989307-0754-4796-a7f4-79a20accfbd5","Type":"ContainerStarted","Data":"ba6830afd8567063ccff765c08fd4bbc3cc2e495eb8f2f8af91d5bc3051a5f41"}
Oct 30 21:51:44.883604 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:44.882733    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5" event={"ID":"a5989307-0754-4796-a7f4-79a20accfbd5","Type":"ContainerStarted","Data":"9f652a611a3902b5ee6d673f34c23f9c49e108a2b3cb1d19db313ce25b35d52c"}
Oct 30 21:51:44.884747 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:44.884221    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk" event={"ID":"e00cca75-dfc2-4231-ad1c-61c7069faa38","Type":"ContainerStarted","Data":"965e41df6b94f7294ea27d1d9f3c3c924b5bae9d16993cab243fe45757213f74"}
Oct 30 21:51:44.884804 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:44.884778    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk" event={"ID":"e00cca75-dfc2-4231-ad1c-61c7069faa38","Type":"ContainerStarted","Data":"2b12a3f966b483859004f4baa78482ca55f2a57bfb365c8a00ae51eeb4ef376b"}
Oct 30 21:51:45.001108 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:45.001051    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" podStartSLOduration=2.001036874 podStartE2EDuration="2.001036874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:44.997680851 +0000 UTC m=+1047.669584818" watchObservedRunningTime="2026-10-30 21:51:45.001036874 +0000 UTC m=+1047.672940809"
Oct 30 21:51:45.060257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:45.059586    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:45.072261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:45.063344    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:51:45.072261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:45.063404    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" podUID="146b8994-5857-43e6-a00e-40683229192f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 21:51:45.147316 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:45.120670    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk" podStartSLOduration=2.12065075 podStartE2EDuration="2.12065075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:45.11851562 +0000 UTC m=+1047.790419565" watchObservedRunningTime="2026-10-30 21:51:45.12065075 +0000 UTC m=+1047.792554685"
Oct 30 21:51:45.276258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:45.275256    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5" podStartSLOduration=3.275211433 podStartE2EDuration="3.275211433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:45.25459951 +0000 UTC m=+1047.926503448" watchObservedRunningTime="2026-10-30 21:51:45.275211433 +0000 UTC m=+1047.947115370"
Oct 30 21:51:45.404832 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:45.404798    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:45.405158 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:45.405136    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:51:45.405215 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:45.405187    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5" podUID="a5989307-0754-4796-a7f4-79a20accfbd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:51:45.489312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:45.480330    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:51:45.489312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:45.480403    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk" podUID="e00cca75-dfc2-4231-ad1c-61c7069faa38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:51:45.501225 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:45.496922    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:46.081256 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:46.072362    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:51:46.081256 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:46.072428    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" podUID="146b8994-5857-43e6-a00e-40683229192f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 21:51:46.403856 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:46.403825    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:51:46.404064 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:46.403882    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5" podUID="a5989307-0754-4796-a7f4-79a20accfbd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:51:46.453260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:46.452621    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:51:46.453260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:46.452689    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk" podUID="e00cca75-dfc2-4231-ad1c-61c7069faa38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:51:47.064260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:47.063738    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:51:47.064260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:47.063803    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" podUID="146b8994-5857-43e6-a00e-40683229192f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 21:51:47.202619 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:47.202557    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:47.403973 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:47.403941    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:51:47.404153 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:47.404016    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5" podUID="a5989307-0754-4796-a7f4-79a20accfbd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:51:47.491423 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:47.491376    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:51:47.491633 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:47.491462    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk" podUID="e00cca75-dfc2-4231-ad1c-61c7069faa38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:51:47.941260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:47.936701    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:48.070353 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:48.069767    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:51:48.070353 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:48.069842    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" podUID="146b8994-5857-43e6-a00e-40683229192f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 21:51:48.403480 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:48.403387    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:51:48.403639 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:48.403478    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5" podUID="a5989307-0754-4796-a7f4-79a20accfbd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:51:48.408358 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:51:48.408322    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:51:48.465259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:48.463132    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:51:48.465259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:48.463243    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk" podUID="e00cca75-dfc2-4231-ad1c-61c7069faa38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:51:49.062257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:49.061680    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:51:49.062257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:49.061749    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" podUID="146b8994-5857-43e6-a00e-40683229192f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 21:51:49.403915 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:49.403889    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:51:49.404395 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:49.404372    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5" podUID="a5989307-0754-4796-a7f4-79a20accfbd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:51:49.454720 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:49.454668    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:51:49.455004 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:49.454972    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk" podUID="e00cca75-dfc2-4231-ad1c-61c7069faa38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:51:50.059702 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:50.059666    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:51:50.060004 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:50.059953    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" podUID="146b8994-5857-43e6-a00e-40683229192f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 21:51:50.403713 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:50.403607    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:51:50.403713 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:50.403673    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5" podUID="a5989307-0754-4796-a7f4-79a20accfbd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:51:50.453621 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:50.453586    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:51:50.454147 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:50.453653    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk" podUID="e00cca75-dfc2-4231-ad1c-61c7069faa38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:51:51.059205 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:51.059171    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 21:51:51.059380 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:51.059225    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" podUID="146b8994-5857-43e6-a00e-40683229192f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 21:51:51.407666 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:51.407635    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:51.408064 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:51.408042    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:51.408745 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:51.408728    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 21:51:51.455189 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:51.455164    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:51.455605 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:51.455444    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:51.456096 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:51.456079    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 21:51:51.724864 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:51.724769    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:52.064223 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:52.064137    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:52.064571 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:52.064543    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:52.066179 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:52.066155    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 21:51:52.213938 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:52.213905    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:57.000214 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:57.000172    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:59.890357 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:59.890324    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"]
Oct 30 21:51:59.890842 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:51:59.890520    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk" podUID="ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" containerName="istio-proxy" containerID="cri-o://a07e6ab6c1da9515a0ff4fc6dc49ac1ecd2ec044c4f7ce3826e0cda645e9e9af" gracePeriod=30
Oct 30 21:52:01.792427 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:01.792388    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:05.164549 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.164517    5586 generic.go:356] "Generic (PLEG): container finished" podID="ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" containerID="a07e6ab6c1da9515a0ff4fc6dc49ac1ecd2ec044c4f7ce3826e0cda645e9e9af" exitCode=0
Oct 30 21:52:05.164916 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.164596    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk" event={"ID":"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb","Type":"ContainerDied","Data":"a07e6ab6c1da9515a0ff4fc6dc49ac1ecd2ec044c4f7ce3826e0cda645e9e9af"}
Oct 30 21:52:05.164916 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.164662    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk" event={"ID":"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb","Type":"ContainerDied","Data":"177bd343aeef6377790ee893e4e620c69a687aaf337ae8df0129672a7c9dee02"}
Oct 30 21:52:05.164916 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.164685    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="177bd343aeef6377790ee893e4e620c69a687aaf337ae8df0129672a7c9dee02"
Oct 30 21:52:05.164916 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.164853    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:52:05.332226 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.332176    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-workload-certs\") pod \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") "
Oct 30 21:52:05.332550 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.332504    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-workload-certs" pod "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" (UID: "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.332753 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.332606    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-envoy\") pod \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") "
Oct 30 21:52:05.332964 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.332938    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-podinfo\") pod \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") "
Oct 30 21:52:05.333056 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.333024    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-workload-socket\") pod \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") "
Oct 30 21:52:05.333123 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.333059    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-token\" (UniqueName: \"kubernetes.io/projected/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-token\") pod \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") "
Oct 30 21:52:05.333175 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.333143    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istiod-ca-cert\") pod \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") "
Oct 30 21:52:05.333226 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.333171    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-kube-api-access-fhvp2\" (UniqueName: \"kubernetes.io/projected/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-kube-api-access-fhvp2\") pod \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") "
Oct 30 21:52:05.333226 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.333204    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-credential-socket\") pod \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") "
Oct 30 21:52:05.333486 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.333400    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-workload-socket" pod "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" (UID: "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.333920 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.333897    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-credential-socket" pod "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" (UID: "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.334046 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.333960    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-data\") pod \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\" (UID: \"ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb\") "
Oct 30 21:52:05.334350 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.334325    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-data" pod "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" (UID: "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.334496 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.334471    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istiod-ca-cert" pod "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" (UID: "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:05.334563 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.334525    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.334563 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.334549    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.334665 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.334563    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.334665 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.334577    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.335734 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.335708    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-envoy" pod "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" (UID: "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:05.335944 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.335918    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-podinfo" pod "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" (UID: "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:52:05.336600 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.336570    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-kube-api-access-fhvp2" pod "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" (UID: "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb"). InnerVolumeSpecName "kube-api-access-fhvp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:05.337698 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.337663    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-token" pod "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" (UID: "ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:05.436144 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.436049    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.436144 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.436094    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.436144 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.436109    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.436144 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.436122    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.436144 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.436139    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhvp2\" (UniqueName: \"kubernetes.io/projected/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb-kube-api-access-fhvp2\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:52:05.461201 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:05.461169    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:06.171485 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:06.171446    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"
Oct 30 21:52:06.403520 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:06.403488    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"]
Oct 30 21:52:06.406203 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:06.406179    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-v4zqk"]
Oct 30 21:52:06.862961 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:06.862917    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:07.450210 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:07.450157    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" path="/var/lib/kubelet/pods/ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb/volumes"
Oct 30 21:52:11.686196 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:11.686155    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:16.486118 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:16.486082    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:17.593495 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:17.593460    5586 scope.go:122] "RemoveContainer" containerID="a07e6ab6c1da9515a0ff4fc6dc49ac1ecd2ec044c4f7ce3826e0cda645e9e9af"
Oct 30 21:52:18.362026 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:52:18.361989    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:52:20.599360 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.599327    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"]
Oct 30 21:52:20.599887 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.599874    5586 cpu_manager.go:414] "containerMap: removing container" podUID="ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" containerName="istio-proxy"
Oct 30 21:52:20.599927 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.599888    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" containerName="istio-proxy"
Oct 30 21:52:20.600000 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.599991    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="ede8564d-fd7e-4cdf-94a2-7983f5d2e1cb" containerName="istio-proxy"
Oct 30 21:52:20.605178 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.605158    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.662997 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.662967    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"]
Oct 30 21:52:20.672994 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.672964    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-tmp-dir\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.673126 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.673096    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e412c89-24a0-474e-9e20-6be984b88f3e-tls-certs\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.673173 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.673129    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g4kr\" (UniqueName: \"kubernetes.io/projected/6e412c89-24a0-474e-9e20-6be984b88f3e-kube-api-access-5g4kr\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.673173 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.673165    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-home\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.673301 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.673192    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-dshm\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.673398 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.673379    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-model-cache\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.708922 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.708889    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-9b51e1\"/\"openshift-service-ca.crt\""
Oct 30 21:52:20.709139 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.709120    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-9b51e1\"/\"kube-root-ca.crt\""
Oct 30 21:52:20.710210 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.710181    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:20.711099 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.711082    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-9b51e1\"/\"default-dockercfg-kt4rt\""
Oct 30 21:52:20.711342 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.711326    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-9b51e1\"/\"distinct-w5-9b51e1-kserve-self-signed-certs\""
Oct 30 21:52:20.776007 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.775971    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-model-cache\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.776271 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.776072    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-tmp-dir\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.776271 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.776163    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e412c89-24a0-474e-9e20-6be984b88f3e-tls-certs\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.776271 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.776196    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g4kr\" (UniqueName: \"kubernetes.io/projected/6e412c89-24a0-474e-9e20-6be984b88f3e-kube-api-access-5g4kr\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.776271 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.776253    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-home\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.777093 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.776290    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-dshm\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.777093 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.776471    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-model-cache\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.777093 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.777016    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-tmp-dir\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.777400 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.777373    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-home\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.779635 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.779608    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e412c89-24a0-474e-9e20-6be984b88f3e-tls-certs\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.780478 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.780454    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-dshm\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.839152 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.839122    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g4kr\" (UniqueName: \"kubernetes.io/projected/6e412c89-24a0-474e-9e20-6be984b88f3e-kube-api-access-5g4kr\") pod \"distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:20.917862 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:20.917817    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:21.319872 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:21.319846    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"]
Oct 30 21:52:21.320826 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:52:21.320791    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e412c89_24a0_474e_9e20_6be984b88f3e.slice/crio-d9940845a1487f6f4447cb638ca9cba2c0dda7029a4b96cf5f44b854565fd523 WatchSource:0}: Error finding container d9940845a1487f6f4447cb638ca9cba2c0dda7029a4b96cf5f44b854565fd523: Status 404 returned error can't find the container with id d9940845a1487f6f4447cb638ca9cba2c0dda7029a4b96cf5f44b854565fd523
Oct 30 21:52:22.052146 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:22.052113    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:22.308614 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:22.308509    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" event={"ID":"6e412c89-24a0-474e-9e20-6be984b88f3e","Type":"ContainerStarted","Data":"affa215878658387953b994491a29506ff0ada3fe72211fe8501492bf60d4838"}
Oct 30 21:52:22.308614 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:22.308558    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" event={"ID":"6e412c89-24a0-474e-9e20-6be984b88f3e","Type":"ContainerStarted","Data":"d9940845a1487f6f4447cb638ca9cba2c0dda7029a4b96cf5f44b854565fd523"}
Oct 30 21:52:22.918108 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:22.918072    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:26.186073 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:26.186036    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:29.479340 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.479203    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podStartSLOduration=9.479187743 podStartE2EDuration="9.479187743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:22.427468872 +0000 UTC m=+1085.099372808" watchObservedRunningTime="2026-10-30 21:52:29.479187743 +0000 UTC m=+1092.151091677"
Oct 30 21:52:29.479931 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.479882    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"]
Oct 30 21:52:29.483865 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.483843    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.571113 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.571080    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"]
Oct 30 21:52:29.584906 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.584879    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23db1e9a-3c13-4c53-9c92-028efebcec23-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.585079 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.584935    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.585079 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.584958    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.585161 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.585090    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.585161 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.585118    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.585161 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.585146    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.585357 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.585336    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.585422 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.585370    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rksnh\" (UniqueName: \"kubernetes.io/projected/23db1e9a-3c13-4c53-9c92-028efebcec23-kube-api-access-rksnh\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.585482 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.585468    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.686122 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.686083    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.686122 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.686122    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.686404 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.686162    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.686404 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.686287    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.686404 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.686317    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rksnh\" (UniqueName: \"kubernetes.io/projected/23db1e9a-3c13-4c53-9c92-028efebcec23-kube-api-access-rksnh\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.686404 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.686368    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.686601 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.686443    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23db1e9a-3c13-4c53-9c92-028efebcec23-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.686601 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.686525    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.686601 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.686548    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.686793 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.686666    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.686852 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.686804    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.686919 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.686894    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.687008 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.686986    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.687192 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.687168    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23db1e9a-3c13-4c53-9c92-028efebcec23-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.688576 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.688560    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.688797 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.688780    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.745047 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.744973    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.745047 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.745010    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rksnh\" (UniqueName: \"kubernetes.io/projected/23db1e9a-3c13-4c53-9c92-028efebcec23-kube-api-access-rksnh\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:29.798593 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:29.798555    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:30.228732 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:30.228701    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"]
Oct 30 21:52:30.229580 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:52:30.229555    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23db1e9a_3c13_4c53_9c92_028efebcec23.slice/crio-cbcd033ff337cb835e4cd8e8f7fab19cb3597c8badc5020503316c186eb61211 WatchSource:0}: Error finding container cbcd033ff337cb835e4cd8e8f7fab19cb3597c8badc5020503316c186eb61211: Status 404 returned error can't find the container with id cbcd033ff337cb835e4cd8e8f7fab19cb3597c8badc5020503316c186eb61211
Oct 30 21:52:30.231799 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:30.231765    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:52:30.231900 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:30.231840    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:52:30.231900 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:30.231883    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:52:30.364979 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:30.364945    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n" event={"ID":"23db1e9a-3c13-4c53-9c92-028efebcec23","Type":"ContainerStarted","Data":"69aba28005014ce806931e1ebbb4cac213bf7b7ac086d9928236bad04c45cbc4"}
Oct 30 21:52:30.365176 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:30.364995    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n" event={"ID":"23db1e9a-3c13-4c53-9c92-028efebcec23","Type":"ContainerStarted","Data":"cbcd033ff337cb835e4cd8e8f7fab19cb3597c8badc5020503316c186eb61211"}
Oct 30 21:52:30.486290 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:30.486178    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n" podStartSLOduration=1.486160308 podStartE2EDuration="1.486160308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:30.480935549 +0000 UTC m=+1093.152839501" watchObservedRunningTime="2026-10-30 21:52:30.486160308 +0000 UTC m=+1093.158064242"
Oct 30 21:52:30.799668 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:30.799585    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:30.896692 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:30.896659    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:30.918163 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:30.918132    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:30.981059 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:30.981029    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:31.246570 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:31.246527    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:31.382451 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:31.382422    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:52:31.801129 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:31.801095    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:52:31.801666 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:31.801161    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n" podUID="23db1e9a-3c13-4c53-9c92-028efebcec23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 21:52:32.799835 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:32.799791    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:52:32.800036 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:32.799873    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n" podUID="23db1e9a-3c13-4c53-9c92-028efebcec23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 21:52:33.800054 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:33.800017    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:52:33.800617 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:33.800125    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n" podUID="23db1e9a-3c13-4c53-9c92-028efebcec23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 21:52:34.799496 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:34.799449    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:52:34.799701 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:34.799529    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n" podUID="23db1e9a-3c13-4c53-9c92-028efebcec23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 21:52:35.799322 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:35.799281    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:52:35.799834 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:35.799361    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n" podUID="23db1e9a-3c13-4c53-9c92-028efebcec23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 21:52:36.099292 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:36.099179    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:36.799328 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:36.799284    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:52:36.799762 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:36.799356    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n" podUID="23db1e9a-3c13-4c53-9c92-028efebcec23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 21:52:37.803659 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:37.803625    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:37.804071 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:37.803918    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:37.804611 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:37.804593    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 21:52:40.868493 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:40.868455    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:45.673811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:45.673774    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:48.380274 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:52:48.379973    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:52:50.470212 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:50.470169    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:54.146520 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:54.146480    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:55.267580 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:55.267544    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:59.042356 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.042319    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"]
Oct 30 21:52:59.046976 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.046953    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.141531 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.141500    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"]
Oct 30 21:52:59.174904 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.174865    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.174904 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.174907    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.175123 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.174942    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.175123 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.175102    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.175199 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.175143    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.175249 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.175200    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9drv\" (UniqueName: \"kubernetes.io/projected/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-kube-api-access-d9drv\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.175289 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.175266    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.175289 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.175286    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.175376 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.175352    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.276103 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.276060    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.276312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.276209    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.276312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.276257    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.276312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.276295    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.276505 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.276387    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.276505 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.276453    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.276608 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.276450    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.276608 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.276505    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9drv\" (UniqueName: \"kubernetes.io/projected/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-kube-api-access-d9drv\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.276713 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.276625    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.276713 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.276645    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.276713 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.276660    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.276861 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.276729    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.277032 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.277006    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.277107 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.277067    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.278796 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.278775    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.279134 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.279112    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.333421 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.333348    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.333640 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.333617    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9drv\" (UniqueName: \"kubernetes.io/projected/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-kube-api-access-d9drv\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.361397 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.361365    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:52:59.795735 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.795708    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"]
Oct 30 21:52:59.797248 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:52:59.797207    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ff1f09e_67cd_4c5c_9831_75c9dd76aa5a.slice/crio-52b89a6831aa88c6f33ab104d54f090fb292ee784d48b78d7a77d517799cc8e7 WatchSource:0}: Error finding container 52b89a6831aa88c6f33ab104d54f090fb292ee784d48b78d7a77d517799cc8e7: Status 404 returned error can't find the container with id 52b89a6831aa88c6f33ab104d54f090fb292ee784d48b78d7a77d517799cc8e7
Oct 30 21:52:59.799675 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.799644    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:52:59.799747 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.799708    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:52:59.799747 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:52:59.799735    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:53:00.085618 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:00.085522    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:00.580122 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:00.578461    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w" event={"ID":"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a","Type":"ContainerStarted","Data":"83c1121e575c1f56c2cb1e3c89536fed17a3828d8259b0ed045764d1e3d1b165"}
Oct 30 21:53:00.580122 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:00.578584    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w" event={"ID":"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a","Type":"ContainerStarted","Data":"52b89a6831aa88c6f33ab104d54f090fb292ee784d48b78d7a77d517799cc8e7"}
Oct 30 21:53:00.752715 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:00.752644    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w" podStartSLOduration=2.752623748 podStartE2EDuration="2.752623748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:00.749469191 +0000 UTC m=+1123.421373129" watchObservedRunningTime="2026-10-30 21:53:00.752623748 +0000 UTC m=+1123.424527686"
Oct 30 21:53:01.361816 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:01.361778    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:53:01.363777 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:01.363751    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:53:01.363906 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:01.363810    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w" podUID="0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 21:53:02.361813 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:02.361776    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:53:02.362001 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:02.361842    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w" podUID="0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 21:53:03.362765 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:03.362723    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:53:03.363304 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:03.362791    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w" podUID="0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 21:53:04.362550 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:04.362510    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:53:04.362751 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:04.362573    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w" podUID="0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 21:53:04.835483 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:04.835444    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:04.906843 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:04.906809    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"]
Oct 30 21:53:04.910798 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:04.910775    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.011821 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.011779    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-80c366\"/\"distinct-w1-80c366-kserve-self-signed-certs\""
Oct 30 21:53:05.015173 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.015144    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"]
Oct 30 21:53:05.047738 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.047683    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:05.073510 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.066109    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-dshm\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.075732 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.075038    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-home\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.075732 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.075142    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-tls-certs\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.075732 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.075190    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-tmp-dir\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.075732 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.075283    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9gct\" (UniqueName: \"kubernetes.io/projected/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-kube-api-access-k9gct\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.075732 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.075361    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-model-cache\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.177163 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.177112    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-model-cache\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.177416 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.177226    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-dshm\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.177416 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.177293    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-home\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.177416 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.177326    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-tls-certs\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.177416 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.177352    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-tmp-dir\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.177416 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.177407    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9gct\" (UniqueName: \"kubernetes.io/projected/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-kube-api-access-k9gct\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.177740 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.177704    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-home\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.177822 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.177804    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-tmp-dir\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.177894 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.177880    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-model-cache\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.179681 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.179654    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-dshm\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.179934 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.179911    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-tls-certs\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.246821 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.246740    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9gct\" (UniqueName: \"kubernetes.io/projected/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-kube-api-access-k9gct\") pod \"distinct-w1-80c366-kserve-756cddd86b-w6vm2\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") " pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.250632 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.250597    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:05.368275 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.365711    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:53:05.368275 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.365773    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w" podUID="0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 21:53:05.701510 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:05.701475    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"]
Oct 30 21:53:05.703106 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:53:05.703075    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda63b784e_dd7f_4c97_b1f2_40644ef3d5e6.slice/crio-cc93810ef710876237c6e74b680b5313f413d96addf4952e5ce39a9fc77fe5f0 WatchSource:0}: Error finding container cc93810ef710876237c6e74b680b5313f413d96addf4952e5ce39a9fc77fe5f0: Status 404 returned error can't find the container with id cc93810ef710876237c6e74b680b5313f413d96addf4952e5ce39a9fc77fe5f0
Oct 30 21:53:06.362542 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:06.362498    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 21:53:06.362988 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:06.362577    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w" podUID="0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 21:53:06.635316 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:06.634172    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" event={"ID":"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6","Type":"ContainerStarted","Data":"5d2b4999d29de02d8d493c74e5d5833c4405d5cb78963897853aa4a94bbdac9c"}
Oct 30 21:53:06.635316 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:06.634224    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" event={"ID":"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6","Type":"ContainerStarted","Data":"cc93810ef710876237c6e74b680b5313f413d96addf4952e5ce39a9fc77fe5f0"}
Oct 30 21:53:06.761727 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:06.761653    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podStartSLOduration=2.761628192 podStartE2EDuration="2.761628192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:06.756751508 +0000 UTC m=+1129.428655448" watchObservedRunningTime="2026-10-30 21:53:06.761628192 +0000 UTC m=+1129.433532127"
Oct 30 21:53:07.250774 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:07.250679    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:07.368226 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:07.368188    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:53:07.369681 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:07.369656    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:53:07.370487 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:07.370464    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 21:53:09.660668 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:09.660632    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:12.917004 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:12.916970    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"]
Oct 30 21:53:12.921211 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:12.921187    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:12.978542 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:12.978502    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:12.978723 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:12.978556    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9gs7\" (UniqueName: \"kubernetes.io/projected/832ac4dd-f2fc-47b3-82f6-03253f91ff46-kube-api-access-f9gs7\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:12.978723 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:12.978638    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:12.978723 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:12.978678    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:12.978834 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:12.978761    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:12.978834 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:12.978817    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:12.978899 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:12.978864    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:12.978941 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:12.978915    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:12.979036 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:12.979013    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.024592 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.024556    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"]
Oct 30 21:53:13.080371 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.080338    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.080568 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.080411    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.080568 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.080470    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.080568 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.080502    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.080736 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.080572    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.080736 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.080615    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9gs7\" (UniqueName: \"kubernetes.io/projected/832ac4dd-f2fc-47b3-82f6-03253f91ff46-kube-api-access-f9gs7\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.080736 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.080663    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.080736 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.080703    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.080736 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.080722    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.080968 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.080845    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.080968 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.080938    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.081068 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.081050    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.081206 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.081188    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.081403 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.081360    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.082865 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.082843    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.083210 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.083190    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.144489 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.144453    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9gs7\" (UniqueName: \"kubernetes.io/projected/832ac4dd-f2fc-47b3-82f6-03253f91ff46-kube-api-access-f9gs7\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.144930 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.144906    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.235441 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.235358    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:13.667909 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:53:13.667876    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod832ac4dd_f2fc_47b3_82f6_03253f91ff46.slice/crio-e6c12fbfd75b00e18e66a917d5bf9b8753ef9253c280c6808253a3703157b04a WatchSource:0}: Error finding container e6c12fbfd75b00e18e66a917d5bf9b8753ef9253c280c6808253a3703157b04a: Status 404 returned error can't find the container with id e6c12fbfd75b00e18e66a917d5bf9b8753ef9253c280c6808253a3703157b04a
Oct 30 21:53:13.670340 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.670308    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:53:13.670433 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.670377    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:53:13.670433 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.670408    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:53:13.671315 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.671293    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"]
Oct 30 21:53:13.688111 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:13.688082    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r" event={"ID":"832ac4dd-f2fc-47b3-82f6-03253f91ff46","Type":"ContainerStarted","Data":"e6c12fbfd75b00e18e66a917d5bf9b8753ef9253c280c6808253a3703157b04a"}
Oct 30 21:53:14.395136 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:14.395095    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:14.696159 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:14.696122    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r" event={"ID":"832ac4dd-f2fc-47b3-82f6-03253f91ff46","Type":"ContainerStarted","Data":"10bdbd357e3d1fb12925b211cb1ad66f9802070ecc645646207de6798d409289"}
Oct 30 21:53:14.862515 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:14.862454    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r" podStartSLOduration=2.862437985 podStartE2EDuration="2.862437985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:14.860776235 +0000 UTC m=+1137.532680170" watchObservedRunningTime="2026-10-30 21:53:14.862437985 +0000 UTC m=+1137.534341991"
Oct 30 21:53:15.245128 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:15.242607    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:15.245491 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:15.245464    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:53:15.245662 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:15.245633    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r" podUID="832ac4dd-f2fc-47b3-82f6-03253f91ff46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 21:53:15.251386 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:15.251361    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:15.263650 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:15.263630    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:15.714306 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:15.714280    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:53:16.241771 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:16.241733    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:53:16.242032 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:16.241997    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r" podUID="832ac4dd-f2fc-47b3-82f6-03253f91ff46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 21:53:17.239256 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:17.238680    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:53:17.239256 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:17.238767    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r" podUID="832ac4dd-f2fc-47b3-82f6-03253f91ff46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 21:53:18.242805 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:18.242761    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:53:18.243481 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:18.243442    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r" podUID="832ac4dd-f2fc-47b3-82f6-03253f91ff46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 21:53:18.375811 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:53:18.375772    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:53:18.377709 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:18.377686    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:18.541931 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:18.541836    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:19.211954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:19.211914    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:19.240744 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:19.240702    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:53:19.241100 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:19.241060    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r" podUID="832ac4dd-f2fc-47b3-82f6-03253f91ff46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 21:53:20.236157 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:20.236108    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 21:53:20.236605 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:20.236184    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r" podUID="832ac4dd-f2fc-47b3-82f6-03253f91ff46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 21:53:21.241023 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:21.240990    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:21.241432 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:21.241329    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:21.242161 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:21.242139    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 21:53:21.608357 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:21.608221    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:24.008192 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:24.008154    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:28.882854 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:28.882823    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:33.674886 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:33.674848    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:38.694963 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:38.694913    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:43.490491 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:43.490451    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:43.708581 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:43.708521    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:48.325464 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:48.325413    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:48.382362 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:53:48.382324    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:53:53.081769 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:53.081726    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:57.888182 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:53:57.888148    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:02.678026 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:02.677939    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:07.476464 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:07.476417    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:12.377297 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:12.377255    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:14.919709 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:14.919668    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"]
Oct 30 21:54:14.923750 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:14.923731    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.076319 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.076278    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7j6d\" (UniqueName: \"kubernetes.io/projected/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-kube-api-access-l7j6d\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.076501 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.076363    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.076501 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.076450    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.076576 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.076531    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.076576 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.076556    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.076576 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.076573    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.076722 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.076630    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.076722 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.076675    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.076830 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.076779    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.137800 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.137763    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"]
Oct 30 21:54:15.178442 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.178347    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.178442 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.178397    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.178442 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.178430    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.178745 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.178458    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.178745 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.178494    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.178745 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.178520    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.178745 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.178565    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.178745 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.178736    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7j6d\" (UniqueName: \"kubernetes.io/projected/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-kube-api-access-l7j6d\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.179035 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.178768    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.179035 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.178799    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.179035 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.178847    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.179219 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.179113    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.179219 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.179160    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.179587 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.179555    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.181056 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.181036    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.181292 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.181271    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.236729 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.236701    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.236893 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.236852    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7j6d\" (UniqueName: \"kubernetes.io/projected/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-kube-api-access-l7j6d\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.236949 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.236935    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:15.676587 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.676547    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"]
Oct 30 21:54:15.713570 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:54:15.713534    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eba1a2d_9bc2_4a7e_9840_e3071d7cf0f3.slice/crio-e816a22aa8f118bc00d5bb6338c93cd4b57b7320a842e58c606197dee38bb9b9 WatchSource:0}: Error finding container e816a22aa8f118bc00d5bb6338c93cd4b57b7320a842e58c606197dee38bb9b9: Status 404 returned error can't find the container with id e816a22aa8f118bc00d5bb6338c93cd4b57b7320a842e58c606197dee38bb9b9
Oct 30 21:54:15.716136 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.716103    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:54:15.716215 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.716171    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:54:15.716215 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:15.716198    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:54:16.130851 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:16.130742    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" event={"ID":"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3","Type":"ContainerStarted","Data":"cfbfed66ebe801b59031c5fbdb440fd36351dea81229e09ec5b1d675d544f147"}
Oct 30 21:54:16.130851 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:16.130846    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" event={"ID":"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3","Type":"ContainerStarted","Data":"e816a22aa8f118bc00d5bb6338c93cd4b57b7320a842e58c606197dee38bb9b9"}
Oct 30 21:54:16.237688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:16.237652    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:16.249495 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:16.249456    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:54:16.249699 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:16.249525    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" podUID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 21:54:16.260564 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:16.260491    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" podStartSLOduration=2.260472818 podStartE2EDuration="2.260472818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:16.258939222 +0000 UTC m=+1198.930843158" watchObservedRunningTime="2026-10-30 21:54:16.260472818 +0000 UTC m=+1198.932376754"
Oct 30 21:54:17.246661 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:17.246623    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:54:17.247168 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:17.246917    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" podUID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 21:54:17.480819 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:17.480780    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:17.618248 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:17.618144    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 21:54:17.618395 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:17.618375    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 21:54:17.649856 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:17.649826    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 21:54:17.650408 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:17.650384    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 21:54:18.242259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:18.241742    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:54:18.242259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:18.241817    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" podUID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 21:54:18.385147 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:54:18.385102    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:54:18.551461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:18.551359    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:19.240259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:19.238051    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:54:19.240259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:19.238115    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" podUID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 21:54:20.237714 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:20.237669    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:54:20.238347 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:20.238311    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" podUID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 21:54:21.246672 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:21.246635    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:54:21.247125 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:21.246696    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" podUID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 21:54:21.948123 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:21.948084    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:22.237619 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:22.237514    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:54:22.237619 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:22.237583    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" podUID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 21:54:22.493678 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:22.493584    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:23.242282 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:23.242249    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:23.242644 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:23.242624    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:23.243322 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:23.243303    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 21:54:27.412194 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:27.411250    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:28.491409 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.491367    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"]
Oct 30 21:54:28.496754 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.496731    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.552449 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.552417    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"]
Oct 30 21:54:28.576883 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.576832    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-model-cache\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.576883 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.576874    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9jwh\" (UniqueName: \"kubernetes.io/projected/2eebc209-0c68-4e4c-9485-466eb87d6154-kube-api-access-q9jwh\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.577141 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.576945    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-tmp-dir\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.577141 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.577084    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-dshm\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.577141 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.577119    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eebc209-0c68-4e4c-9485-466eb87d6154-tls-certs\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.577393 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.577168    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-home\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.597489 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.597452    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-9b51e1\"/\"distinct-2-w5-9b51e1-kserve-self-signed-certs\""
Oct 30 21:54:28.615446 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.615400    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:28.678917 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.678878    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-model-cache\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.678917 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.678918    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9jwh\" (UniqueName: \"kubernetes.io/projected/2eebc209-0c68-4e4c-9485-466eb87d6154-kube-api-access-q9jwh\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.679196 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.678947    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-tmp-dir\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.679196 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.679066    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-dshm\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.679196 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.679085    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eebc209-0c68-4e4c-9485-466eb87d6154-tls-certs\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.679196 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.679114    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-home\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.679444 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.679387    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-model-cache\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.679505 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.679448    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-tmp-dir\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.679555 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.679541    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-home\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.681565 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.681534    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-dshm\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.681811 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.681794    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eebc209-0c68-4e4c-9485-466eb87d6154-tls-certs\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.760773 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.760690    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9jwh\" (UniqueName: \"kubernetes.io/projected/2eebc209-0c68-4e4c-9485-466eb87d6154-kube-api-access-q9jwh\") pod \"distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") " pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.800778 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.800737    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"]
Oct 30 21:54:28.806136 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.806112    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:28.810624 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.810602    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:28.882108 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.882064    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:28.882272 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.882124    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:28.882272 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.882175    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:28.882407 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.882268    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:28.882407 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.882367    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:28.882511 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.882478    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:28.882571 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.882544    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:28.882625 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.882602    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68fz6\" (UniqueName: \"kubernetes.io/projected/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-kube-api-access-68fz6\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:28.882675 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.882655    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:28.907266 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:28.904698    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"]
Oct 30 21:54:29.029305 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.028886    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.029305 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.028996    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.029305 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.029085    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.029305 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.029140    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.029305 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.029206    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68fz6\" (UniqueName: \"kubernetes.io/projected/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-kube-api-access-68fz6\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.029305 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.029272    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.029731 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.029341    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.029731 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.029364    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.029731 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.029406    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.029885 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.029820    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.041120 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.031093    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.041120 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.031191    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.041120 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.031377    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.041120 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.031481    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.048856 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.047534    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.049311 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.049255    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.111313 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.104511    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68fz6\" (UniqueName: \"kubernetes.io/projected/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-kube-api-access-68fz6\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.111313 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.104965    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.158258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.151987    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:29.277260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.271500    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"]
Oct 30 21:54:29.804621 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.804589    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"]
Oct 30 21:54:29.830339 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.830295    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:54:29.830634 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.830606    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:54:29.851936 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:29.830787    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:54:30.363074 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:30.363029    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" event={"ID":"5568d46b-ed6b-4c97-99e0-8a0de7cc5258","Type":"ContainerStarted","Data":"33b13ede262c3a0dd11167439c64cfc3d71b607c071c630294727332f4266288"}
Oct 30 21:54:30.386180 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:30.385130    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" event={"ID":"2eebc209-0c68-4e4c-9485-466eb87d6154","Type":"ContainerStarted","Data":"e23d8d6f66078da4de7aa5c6c716a7bcaf032fea1aec5caa2809de3b91b163b7"}
Oct 30 21:54:30.386180 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:30.385177    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" event={"ID":"2eebc209-0c68-4e4c-9485-466eb87d6154","Type":"ContainerStarted","Data":"b6568755565d57aff2df8536aa31eb88a2ecbde8ff5460dbe876c0941c168d32"}
Oct 30 21:54:30.585587 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:30.556449    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podStartSLOduration=2.55642463 podStartE2EDuration="2.55642463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:30.529043906 +0000 UTC m=+1213.200947843" watchObservedRunningTime="2026-10-30 21:54:30.55642463 +0000 UTC m=+1213.228328566"
Oct 30 21:54:30.823049 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:30.823012    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:31.393481 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:31.393443    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" event={"ID":"5568d46b-ed6b-4c97-99e0-8a0de7cc5258","Type":"ContainerStarted","Data":"c13342f0489dbdc044381f97127790e3f503c4c978f4a7c77d58a31d1247eff8"}
Oct 30 21:54:31.646321 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:31.644737    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" podStartSLOduration=3.6447155430000002 podStartE2EDuration="3.644715543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:31.573144516 +0000 UTC m=+1214.245048451" watchObservedRunningTime="2026-10-30 21:54:31.644715543 +0000 UTC m=+1214.316619480"
Oct 30 21:54:32.153201 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:32.153155    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:32.154956 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:32.154927    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:54:32.155116 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:32.155001    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" podUID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:54:32.660280 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:32.652808    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:33.153743 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:33.153714    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:54:33.154264 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:33.153769    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" podUID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:54:34.153392 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:34.153354    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:54:34.153583 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:34.153412    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" podUID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:54:35.154023 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:35.153983    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:54:35.154506 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:35.154038    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" podUID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:54:36.153454 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:36.153416    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:54:36.153669 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:36.153483    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" podUID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:54:37.055701 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:37.055659    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"]
Oct 30 21:54:37.087153 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:37.087118    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b857e984-lzbv6"]
Oct 30 21:54:37.153621 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:37.153580    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:54:37.153800 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:37.153653    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" podUID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:54:37.400915 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:37.400821    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:37.450719 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:37.450675    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1" path="/var/lib/kubelet/pods/c9e6bc7d-8a41-43e7-aa99-efcebee0f2a1/volumes"
Oct 30 21:54:37.597372 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:37.597335    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:38.286408 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:38.286287    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:38.287120 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:38.287061    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:38.288217 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:38.288193    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 21:54:38.811363 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:38.811329    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:38.824381 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:38.824355    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:39.491159 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:39.491126    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:54:42.307318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:42.307282    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:45.243888 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.243853    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"]
Oct 30 21:54:45.248290 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.248270    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.286319 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.286280    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.286492 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.286391    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.286534 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.286480    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbade6b6-381e-4034-a3c7-075ce93c8218-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.286534 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.286519    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.286605 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.286567    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.286638 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.286630    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.286727 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.286711    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsrsj\" (UniqueName: \"kubernetes.io/projected/cbade6b6-381e-4034-a3c7-075ce93c8218-kube-api-access-vsrsj\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.286764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.286740    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.286800 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.286780    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.387570 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.387529    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbade6b6-381e-4034-a3c7-075ce93c8218-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.387752 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.387578    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.387752 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.387621    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.387752 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.387680    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.387752 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.387698    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsrsj\" (UniqueName: \"kubernetes.io/projected/cbade6b6-381e-4034-a3c7-075ce93c8218-kube-api-access-vsrsj\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.387752 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.387723    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.388012 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.387790    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.388012 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.387875    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.388012 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.387997    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.388228 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.388038    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.388228 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.388093    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.388228 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.388202    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.388429 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.388362    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbade6b6-381e-4034-a3c7-075ce93c8218-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.388429 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.388402    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.390083 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.390058    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.390418 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.390398    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.445981 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.445943    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsrsj\" (UniqueName: \"kubernetes.io/projected/cbade6b6-381e-4034-a3c7-075ce93c8218-kube-api-access-vsrsj\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.446156 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.446098    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.454211 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.454183    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"]
Oct 30 21:54:45.562982 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.562893    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:45.995616 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.995585    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"]
Oct 30 21:54:45.996932 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:54:45.996898    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbade6b6_381e_4034_a3c7_075ce93c8218.slice/crio-e9949fe192a2e0f8aabd17b6ea84a4a72ca0574288726937f7e70140df5d8fb0 WatchSource:0}: Error finding container e9949fe192a2e0f8aabd17b6ea84a4a72ca0574288726937f7e70140df5d8fb0: Status 404 returned error can't find the container with id e9949fe192a2e0f8aabd17b6ea84a4a72ca0574288726937f7e70140df5d8fb0
Oct 30 21:54:45.999370 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.999337    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:54:45.999456 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.999406    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:54:45.999456 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:45.999436    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:54:46.537906 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:46.537778    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr" event={"ID":"cbade6b6-381e-4034-a3c7-075ce93c8218","Type":"ContainerStarted","Data":"d12812762a2d73874883ab158b5a51bdb53c24069a5ae1bf3e48beb524a1dcad"}
Oct 30 21:54:46.537906 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:46.537864    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr" event={"ID":"cbade6b6-381e-4034-a3c7-075ce93c8218","Type":"ContainerStarted","Data":"e9949fe192a2e0f8aabd17b6ea84a4a72ca0574288726937f7e70140df5d8fb0"}
Oct 30 21:54:46.563782 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:46.563747    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:46.714784 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:46.714717    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr" podStartSLOduration=1.714701748 podStartE2EDuration="1.714701748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:46.710163161 +0000 UTC m=+1229.382067103" watchObservedRunningTime="2026-10-30 21:54:46.714701748 +0000 UTC m=+1229.386605683"
Oct 30 21:54:47.155873 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:47.155567    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:47.337131 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:47.336740    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:47.583789 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:47.583328    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:54:47.583789 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:47.583400    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr" podUID="cbade6b6-381e-4034-a3c7-075ce93c8218" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:54:48.374343 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:54:48.374309    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:54:48.563386 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:48.563354    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:54:48.563564 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:48.563417    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr" podUID="cbade6b6-381e-4034-a3c7-075ce93c8218" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:54:49.563521 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:49.563479    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:54:49.564023 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:49.563545    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr" podUID="cbade6b6-381e-4034-a3c7-075ce93c8218" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:54:50.564114 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:50.564071    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:54:50.564664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:50.564135    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr" podUID="cbade6b6-381e-4034-a3c7-075ce93c8218" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:54:51.563688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:51.563651    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:54:51.563890 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:51.563707    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr" podUID="cbade6b6-381e-4034-a3c7-075ce93c8218" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:54:52.221858 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:52.221819    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:52.563628 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:52.563538    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:54:52.563783 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:52.563619    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr" podUID="cbade6b6-381e-4034-a3c7-075ce93c8218" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:54:53.567906 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:53.567874    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:53.568310 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:53.568159    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:53.568860 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:53.568844    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 21:54:56.967530 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:54:56.967488    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:00.256663 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.256626    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"]
Oct 30 21:55:00.261314 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.261289    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.435262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.427827    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.435262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.427958    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.435262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.427996    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wh2d\" (UniqueName: \"kubernetes.io/projected/8310bcd1-2853-49f2-9618-0ff100c794bd-kube-api-access-6wh2d\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.435262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.428043    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.435262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.428073    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.435262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.428104    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.435262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.428132    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.435262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.428169    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.435262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.428306    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8310bcd1-2853-49f2-9618-0ff100c794bd-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.537026 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.536917    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"]
Oct 30 21:55:00.537311 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.537103    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.537552 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.537525    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-workload-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.537677 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.537660    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.537824 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.537803    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wh2d\" (UniqueName: \"kubernetes.io/projected/8310bcd1-2853-49f2-9618-0ff100c794bd-kube-api-access-6wh2d\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.537958 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.537941    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.538082 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.538066    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.538187 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.538175    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.538338 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.538318    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.538415 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.538381    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.538525 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.538504    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8310bcd1-2853-49f2-9618-0ff100c794bd-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.539455 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.539423    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8310bcd1-2853-49f2-9618-0ff100c794bd-istiod-ca-cert\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.539717 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.539691    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-data\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.540376 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.540349    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-credential-socket\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.540963 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.540931    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-workload-certs\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.554931 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.553787    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-envoy\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.559408 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.559330    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-podinfo\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.602257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.599134    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wh2d\" (UniqueName: \"kubernetes.io/projected/8310bcd1-2853-49f2-9618-0ff100c794bd-kube-api-access-6wh2d\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.617357 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.617251    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-token\") pod \"e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") " pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:00.878834 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:00.878730    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:02.219960 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:02.188951    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:02.280069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:02.280015    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:02.307978 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:02.307919    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:02.308870 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:02.304600    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:02.572570 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:55:02.572528    5586 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.13s"
Oct 30 21:55:02.573972 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:02.573942    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"]
Oct 30 21:55:02.716518 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:02.716455    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:02.777762 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:02.776974    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:02.790388 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:02.790267    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:02.922098 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:02.922018    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" event={"ID":"8310bcd1-2853-49f2-9618-0ff100c794bd","Type":"ContainerStarted","Data":"ed3dc2fa25e15612c083835749f23e2c906f9ba1bd36614d802240350967b16c"}
Oct 30 21:55:03.314270 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:03.313744    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:03.314854 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:03.314579    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:55:03.593439 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:03.562962    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:03.795931 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:03.792347    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:03.897965 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:03.895344    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:55:03.941413 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:03.939304    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:55:04.078919 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:04.078695    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:55:04.087168 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:04.087133    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" event={"ID":"8310bcd1-2853-49f2-9618-0ff100c794bd","Type":"ContainerStarted","Data":"001f9ab0b437f7fc4d717fa2ab5b30583ea73c6aca9a3f5fbaa002d8ac7effab"}
Oct 30 21:55:04.592254 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:04.567435    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:04.661785 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:04.656610    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" podStartSLOduration=4.656584311 podStartE2EDuration="4.656584311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:04.431912758 +0000 UTC m=+1247.103816694" watchObservedRunningTime="2026-10-30 21:55:04.656584311 +0000 UTC m=+1247.328488248"
Oct 30 21:55:04.879575 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:04.879481    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:04.880841 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:04.880815    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:55:04.880900 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:04.880869    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" podUID="8310bcd1-2853-49f2-9618-0ff100c794bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:55:05.880794 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:05.880752    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:55:05.881189 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:05.880820    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" podUID="8310bcd1-2853-49f2-9618-0ff100c794bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:55:06.880222 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:06.880189    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:55:06.880390 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:06.880261    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" podUID="8310bcd1-2853-49f2-9618-0ff100c794bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:55:06.989534 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:06.989492    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:07.880423 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:07.880385    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:55:07.880611 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:07.880439    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" podUID="8310bcd1-2853-49f2-9618-0ff100c794bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:55:08.418032 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:08.417998    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:08.879918 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:08.879828    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:55:08.879918 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:08.879881    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" podUID="8310bcd1-2853-49f2-9618-0ff100c794bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:55:09.880702 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:09.880661    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:55:09.881204 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:09.880720    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" podUID="8310bcd1-2853-49f2-9618-0ff100c794bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:55:10.879922 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:10.879879    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:55:10.880102 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:10.879942    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" podUID="8310bcd1-2853-49f2-9618-0ff100c794bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:55:11.885110 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:11.885080    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:11.885577 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:11.885556    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:11.886343 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:11.886323    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 21:55:13.274883 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:13.274850    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:13.888809 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:13.888771    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:18.087785 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:18.087745    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:18.362596 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:55:18.362511    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:55:22.286263 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:22.286214    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:22.912568 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:22.912536    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:24.658467 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.658426    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"]
Oct 30 21:55:24.663961 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.663940    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"
Oct 30 21:55:24.726748 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.726707    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"]
Oct 30 21:55:24.770729 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.770691    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 21:55:24.770919 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.770762    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:55:24.810159 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.810117    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aef30478-1b5f-4cae-b205-ef1364298532-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b48a3910-ppk8r\" (UID: \"aef30478-1b5f-4cae-b205-ef1364298532\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"
Oct 30 21:55:24.810403 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.810194    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9n9l\" (UniqueName: \"kubernetes.io/projected/aef30478-1b5f-4cae-b205-ef1364298532-kube-api-access-g9n9l\") pod \"maas-api-revoke-sub-b48a3910-ppk8r\" (UID: \"aef30478-1b5f-4cae-b205-ef1364298532\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"
Oct 30 21:55:24.810483 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.810444    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aef30478-1b5f-4cae-b205-ef1364298532-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b48a3910-ppk8r\" (UID: \"aef30478-1b5f-4cae-b205-ef1364298532\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"
Oct 30 21:55:24.912272 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.912135    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aef30478-1b5f-4cae-b205-ef1364298532-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b48a3910-ppk8r\" (UID: \"aef30478-1b5f-4cae-b205-ef1364298532\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"
Oct 30 21:55:24.912447 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.912292    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aef30478-1b5f-4cae-b205-ef1364298532-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b48a3910-ppk8r\" (UID: \"aef30478-1b5f-4cae-b205-ef1364298532\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"
Oct 30 21:55:24.912447 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.912360    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9n9l\" (UniqueName: \"kubernetes.io/projected/aef30478-1b5f-4cae-b205-ef1364298532-kube-api-access-g9n9l\") pod \"maas-api-revoke-sub-b48a3910-ppk8r\" (UID: \"aef30478-1b5f-4cae-b205-ef1364298532\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"
Oct 30 21:55:24.912976 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.912952    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aef30478-1b5f-4cae-b205-ef1364298532-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b48a3910-ppk8r\" (UID: \"aef30478-1b5f-4cae-b205-ef1364298532\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"
Oct 30 21:55:24.914594 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.914576    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aef30478-1b5f-4cae-b205-ef1364298532-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b48a3910-ppk8r\" (UID: \"aef30478-1b5f-4cae-b205-ef1364298532\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"
Oct 30 21:55:24.969864 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.969833    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9n9l\" (UniqueName: \"kubernetes.io/projected/aef30478-1b5f-4cae-b205-ef1364298532-kube-api-access-g9n9l\") pod \"maas-api-revoke-sub-b48a3910-ppk8r\" (UID: \"aef30478-1b5f-4cae-b205-ef1364298532\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"
Oct 30 21:55:24.977755 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:24.977697    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"
Oct 30 21:55:25.356746 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:25.356714    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"]
Oct 30 21:55:25.358196 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:55:25.358171    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaef30478_1b5f_4cae_b205_ef1364298532.slice/crio-7d17612d4b3508f32d8fe01140a273a95bbd2d242fd7f5eb641d561c8d27f8ea WatchSource:0}: Error finding container 7d17612d4b3508f32d8fe01140a273a95bbd2d242fd7f5eb641d561c8d27f8ea: Status 404 returned error can't find the container with id 7d17612d4b3508f32d8fe01140a273a95bbd2d242fd7f5eb641d561c8d27f8ea
Oct 30 21:55:26.276824 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:26.276786    5586 generic.go:356] "Generic (PLEG): container finished" podID="aef30478-1b5f-4cae-b205-ef1364298532" containerID="32de16a507b7fd7c8623931b7784bff9fece99c37563df43ed94520cd53bb5f3" exitCode=0
Oct 30 21:55:26.277263 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:26.276868    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r" event={"ID":"aef30478-1b5f-4cae-b205-ef1364298532","Type":"ContainerDied","Data":"32de16a507b7fd7c8623931b7784bff9fece99c37563df43ed94520cd53bb5f3"}
Oct 30 21:55:26.277263 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:26.276923    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r" event={"ID":"aef30478-1b5f-4cae-b205-ef1364298532","Type":"ContainerStarted","Data":"7d17612d4b3508f32d8fe01140a273a95bbd2d242fd7f5eb641d561c8d27f8ea"}
Oct 30 21:55:27.436332 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:27.436304    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"
Oct 30 21:55:27.547875 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:27.547792    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aef30478-1b5f-4cae-b205-ef1364298532-kube-api-access-g9n9l\" (UniqueName: \"kubernetes.io/projected/aef30478-1b5f-4cae-b205-ef1364298532-kube-api-access-g9n9l\") pod \"aef30478-1b5f-4cae-b205-ef1364298532\" (UID: \"aef30478-1b5f-4cae-b205-ef1364298532\") "
Oct 30 21:55:27.548035 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:27.547916    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aef30478-1b5f-4cae-b205-ef1364298532-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aef30478-1b5f-4cae-b205-ef1364298532-maas-api-openshift-ca\") pod \"aef30478-1b5f-4cae-b205-ef1364298532\" (UID: \"aef30478-1b5f-4cae-b205-ef1364298532\") "
Oct 30 21:55:27.548035 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:27.548002    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aef30478-1b5f-4cae-b205-ef1364298532-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aef30478-1b5f-4cae-b205-ef1364298532-maas-api-xks-ca\") pod \"aef30478-1b5f-4cae-b205-ef1364298532\" (UID: \"aef30478-1b5f-4cae-b205-ef1364298532\") "
Oct 30 21:55:27.548341 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:27.548315    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aef30478-1b5f-4cae-b205-ef1364298532-maas-api-openshift-ca" pod "aef30478-1b5f-4cae-b205-ef1364298532" (UID: "aef30478-1b5f-4cae-b205-ef1364298532"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:27.548641 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:27.548621    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aef30478-1b5f-4cae-b205-ef1364298532-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:27.550689 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:27.550653    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aef30478-1b5f-4cae-b205-ef1364298532-kube-api-access-g9n9l" pod "aef30478-1b5f-4cae-b205-ef1364298532" (UID: "aef30478-1b5f-4cae-b205-ef1364298532"). InnerVolumeSpecName "kube-api-access-g9n9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:27.550945 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:27.550916    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aef30478-1b5f-4cae-b205-ef1364298532-maas-api-xks-ca" pod "aef30478-1b5f-4cae-b205-ef1364298532" (UID: "aef30478-1b5f-4cae-b205-ef1364298532"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:27.649748 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:27.649710    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9n9l\" (UniqueName: \"kubernetes.io/projected/aef30478-1b5f-4cae-b205-ef1364298532-kube-api-access-g9n9l\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:27.649748 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:27.649743    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aef30478-1b5f-4cae-b205-ef1364298532-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:27.808367 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:27.808283    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:28.292376 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:28.292341    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r" event={"ID":"aef30478-1b5f-4cae-b205-ef1364298532","Type":"ContainerDied","Data":"7d17612d4b3508f32d8fe01140a273a95bbd2d242fd7f5eb641d561c8d27f8ea"}
Oct 30 21:55:28.292565 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:28.292418    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d17612d4b3508f32d8fe01140a273a95bbd2d242fd7f5eb641d561c8d27f8ea"
Oct 30 21:55:28.292565 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:28.292359    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"
Oct 30 21:55:29.016727 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.016684    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"]
Oct 30 21:55:29.017561 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.017538    5586 cpu_manager.go:414] "containerMap: removing container" podUID="aef30478-1b5f-4cae-b205-ef1364298532" containerName="revoke-keys"
Oct 30 21:55:29.017561 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.017561    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aef30478-1b5f-4cae-b205-ef1364298532" containerName="revoke-keys"
Oct 30 21:55:29.017762 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.017748    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="aef30478-1b5f-4cae-b205-ef1364298532" containerName="revoke-keys"
Oct 30 21:55:29.021526 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.021502    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.066489 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.066452    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.066673 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.066569    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.066673 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.066641    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.066779 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.066720    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.066817 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.066802    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.066859 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.066849    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.066898 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.066880    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.066946 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.066902    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6259p\" (UniqueName: \"kubernetes.io/projected/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-kube-api-access-6259p\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.067067 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.067044    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.124040 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.124004    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"]
Oct 30 21:55:29.167836 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.167787    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.168050 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.167857    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.168050 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.167890    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6259p\" (UniqueName: \"kubernetes.io/projected/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-kube-api-access-6259p\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.168050 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.167945    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.168050 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.167975    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.168279 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.168090    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.168279 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.168124    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.168279 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.168191    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.168490 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.168289    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.168490 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.168368    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-workload-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.168704 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.168676    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-data\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.168858 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.168825    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istiod-ca-cert\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.169309 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.169287    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-workload-certs\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.169418 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.169376    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-credential-socket\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.171163 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.171137    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-envoy\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.171534 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.171501    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-podinfo\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.224806 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.224773    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-token\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.224957 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.224836    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6259p\" (UniqueName: \"kubernetes.io/projected/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-kube-api-access-6259p\") pod \"e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") " pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.335578 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.335494    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:29.570775 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.570737    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"]
Oct 30 21:55:29.576488 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.576461    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.634805 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.634723    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"]
Oct 30 21:55:29.674618 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.674577    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gvbg\" (UniqueName: \"kubernetes.io/projected/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-kube-api-access-9gvbg\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.674815 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.674718    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.674957 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.674928    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.675074 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.675050    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.675136 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.675097    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.675193 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.675168    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.675277 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.675208    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.675344 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.675327    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.675397 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.675387    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.776513 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.776478    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.776700 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.776593    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.776700 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.776641    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.776700 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.776658    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.776700 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.776682    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.776904 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.776730    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.776904 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.776774    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.777008 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.776926    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.777060 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.777045    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gvbg\" (UniqueName: \"kubernetes.io/projected/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-kube-api-access-9gvbg\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.777206 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.777181    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.777308 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.777272    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.777592 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.777563    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.777667 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.777590    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.777667 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.777592    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.779005 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.778981    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.779219 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.779203    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.800286 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.800255    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"]
Oct 30 21:55:29.801111 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:55:29.801078    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf7cdd7_fc8d_4d79_a3e7_cc24f885ccd0.slice/crio-f1c61c0ff1793767eba2aadc91bf3665f8197b711b4cd7fb202d1b8661f1eab4 WatchSource:0}: Error finding container f1c61c0ff1793767eba2aadc91bf3665f8197b711b4cd7fb202d1b8661f1eab4: Status 404 returned error can't find the container with id f1c61c0ff1793767eba2aadc91bf3665f8197b711b4cd7fb202d1b8661f1eab4
Oct 30 21:55:29.804157 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.804124    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:29.804282 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.804198    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:29.804348 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.804286    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:29.833202 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.833169    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.833334 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.833261    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gvbg\" (UniqueName: \"kubernetes.io/projected/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-kube-api-access-9gvbg\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:29.891426 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:29.891347    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:30.199468 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.199436    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"]
Oct 30 21:55:30.204785 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.204760    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.234734 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.234700    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"]
Oct 30 21:55:30.239471 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.239448    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.264985 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.264956    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"]
Oct 30 21:55:30.284639 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.284610    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.284839 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.284670    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xstvs\" (UniqueName: \"kubernetes.io/projected/c3360def-3e78-4997-98af-fefd89f483ee-kube-api-access-xstvs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.285437 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.285384    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.285571 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.285467    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.285571 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.285534    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.285685 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.285646    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.285685 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.285678    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.285972 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.285931    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3360def-3e78-4997-98af-fefd89f483ee-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.286086 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.286006    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dmbw\" (UniqueName: \"kubernetes.io/projected/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-kube-api-access-5dmbw\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.286086 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.286050    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.286256 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.286092    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.286256 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.286123    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.286256 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.286164    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.286256 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.286198    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3360def-3e78-4997-98af-fefd89f483ee-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.286461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.286225    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3360def-3e78-4997-98af-fefd89f483ee-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.286461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.286293    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.286461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.286312    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.286461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.286336    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.321008 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.320962    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" event={"ID":"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0","Type":"ContainerStarted","Data":"d916070f35bd032cea5c5b34265d461c42af8cd2b8a5bfc58dbade6e5fa61032"}
Oct 30 21:55:30.321483 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.321044    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" event={"ID":"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0","Type":"ContainerStarted","Data":"f1c61c0ff1793767eba2aadc91bf3665f8197b711b4cd7fb202d1b8661f1eab4"}
Oct 30 21:55:30.321483 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.321075    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"]
Oct 30 21:55:30.332765 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.332736    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"]
Oct 30 21:55:30.337092 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.336097    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:30.337613 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.337583    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 21:55:30.338607 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.338569    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 21:55:30.343290 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:55:30.343222    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77c5d7c9_2c19_4df8_a2ff_2e159881e1fd.slice/crio-0d749c7f87afeef85c70ca1475a31595ccffbb2119dd26a1a89d564580c62454 WatchSource:0}: Error finding container 0d749c7f87afeef85c70ca1475a31595ccffbb2119dd26a1a89d564580c62454: Status 404 returned error can't find the container with id 0d749c7f87afeef85c70ca1475a31595ccffbb2119dd26a1a89d564580c62454
Oct 30 21:55:30.380812 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.380770    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:30.380941 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.380854    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:30.380941 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.380895    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:30.389669 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.389635    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.389820 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.389715    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.389890 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.389827    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.389890 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.389862    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.389990 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.389909    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3360def-3e78-4997-98af-fefd89f483ee-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.389990 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.389956    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dmbw\" (UniqueName: \"kubernetes.io/projected/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-kube-api-access-5dmbw\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.390138 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.389999    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.390138 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.390030    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.390138 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.390065    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.390138 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.390105    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.390418 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.390143    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3360def-3e78-4997-98af-fefd89f483ee-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.390418 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.390174    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3360def-3e78-4997-98af-fefd89f483ee-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.390418 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.390227    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.390418 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.390274    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.390418 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.390333    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.390418 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.390373    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.390675 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.390445    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xstvs\" (UniqueName: \"kubernetes.io/projected/c3360def-3e78-4997-98af-fefd89f483ee-kube-api-access-xstvs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.390675 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.390499    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.396522 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.393629    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.396522 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.393525    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.396522 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.394150    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.396522 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.394189    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-data\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.396522 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.394515    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-workload-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.396522 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.395022    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.396522 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.395290    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3360def-3e78-4997-98af-fefd89f483ee-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.396522 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.395523    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-workload-certs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.396522 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.395780    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-credential-socket\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.396522 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.396050    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istiod-ca-cert\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.397697 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.397670    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3360def-3e78-4997-98af-fefd89f483ee-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.397697 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.397685    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.398013 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.397740    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-podinfo\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.398563 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.398543    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-istio-envoy\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.444331 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.444269    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" podStartSLOduration=2.444249982 podStartE2EDuration="2.444249982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:30.442751447 +0000 UTC m=+1273.114655395" watchObservedRunningTime="2026-10-30 21:55:30.444249982 +0000 UTC m=+1273.116153910"
Oct 30 21:55:30.453112 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.453075    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.456846 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.456815    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xstvs\" (UniqueName: \"kubernetes.io/projected/c3360def-3e78-4997-98af-fefd89f483ee-kube-api-access-xstvs\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.456976 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.456956    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3360def-3e78-4997-98af-fefd89f483ee-istio-token\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.457277 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.457224    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dmbw\" (UniqueName: \"kubernetes.io/projected/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-kube-api-access-5dmbw\") pod \"e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") " pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.519747 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.519708    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:30.554880 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.554845    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:30.911146 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.911118    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"]
Oct 30 21:55:30.911418 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:55:30.911393    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee1cab02_9630_4c82_b5ed_f3f00e4882ed.slice/crio-799b8b8a3cd2158d73bfdc48e78613a5a000e7c32eed9e727841374914780f43 WatchSource:0}: Error finding container 799b8b8a3cd2158d73bfdc48e78613a5a000e7c32eed9e727841374914780f43: Status 404 returned error can't find the container with id 799b8b8a3cd2158d73bfdc48e78613a5a000e7c32eed9e727841374914780f43
Oct 30 21:55:30.914206 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.914174    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:30.914332 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.914258    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:30.914332 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.914295    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:30.970947 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:55:30.970918    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3360def_3e78_4997_98af_fefd89f483ee.slice/crio-8a37fa4addf772829b514ee77da828bb264f905ec7d3e8f087d5e5b2f1f589ae WatchSource:0}: Error finding container 8a37fa4addf772829b514ee77da828bb264f905ec7d3e8f087d5e5b2f1f589ae: Status 404 returned error can't find the container with id 8a37fa4addf772829b514ee77da828bb264f905ec7d3e8f087d5e5b2f1f589ae
Oct 30 21:55:30.971055 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.971006    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"]
Oct 30 21:55:30.974384 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.974351    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:30.974502 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.974431    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:30.974502 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:30.974477    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:55:31.338341 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.338151    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 21:55:31.338341 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.338218    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 21:55:31.378584 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.378540    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" event={"ID":"c3360def-3e78-4997-98af-fefd89f483ee","Type":"ContainerStarted","Data":"2e9b09c76cae0dfc74829d6a4eaf9929437250dbc16b700bfe92e749d091ea5b"}
Oct 30 21:55:31.378874 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.378850    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" event={"ID":"c3360def-3e78-4997-98af-fefd89f483ee","Type":"ContainerStarted","Data":"8a37fa4addf772829b514ee77da828bb264f905ec7d3e8f087d5e5b2f1f589ae"}
Oct 30 21:55:31.382288 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.382258    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" event={"ID":"ee1cab02-9630-4c82-b5ed-f3f00e4882ed","Type":"ContainerStarted","Data":"d3c71b55f51be831ccd90ab32288ba5bd03e22efb5cd127107bab0c921057496"}
Oct 30 21:55:31.382462 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.382448    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" event={"ID":"ee1cab02-9630-4c82-b5ed-f3f00e4882ed","Type":"ContainerStarted","Data":"799b8b8a3cd2158d73bfdc48e78613a5a000e7c32eed9e727841374914780f43"}
Oct 30 21:55:31.388925 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.388829    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" event={"ID":"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd","Type":"ContainerStarted","Data":"2dd946dde2ce3cd9b36290df05ea6bf79b6994af1421bbf5b7c9afaf6cba5ff5"}
Oct 30 21:55:31.388925 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.388876    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" event={"ID":"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd","Type":"ContainerStarted","Data":"0d749c7f87afeef85c70ca1475a31595ccffbb2119dd26a1a89d564580c62454"}
Oct 30 21:55:31.512446 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.509749    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" podStartSLOduration=1.509730801 podStartE2EDuration="1.509730801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:31.505908524 +0000 UTC m=+1274.177812460" watchObservedRunningTime="2026-10-30 21:55:31.509730801 +0000 UTC m=+1274.181634737"
Oct 30 21:55:31.520023 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.519983    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:31.560263 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.559609    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:31.756969 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.756896    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" podStartSLOduration=2.756872038 podStartE2EDuration="2.756872038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:31.635083298 +0000 UTC m=+1274.306987236" watchObservedRunningTime="2026-10-30 21:55:31.756872038 +0000 UTC m=+1274.428775975"
Oct 30 21:55:31.891880 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.891845    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:31.893465 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.893439    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:55:31.893608 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:31.893490    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:55:32.339760 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:32.339715    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 21:55:32.340394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:32.340358    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 21:55:32.524208 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:32.524167    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:55:32.524520 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:32.524486    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 21:55:32.560119 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:32.560079    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:32.579383 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:32.578985    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:55:32.730261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:32.729396    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:32.896259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:32.896001    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:55:32.896259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:32.896079    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:55:33.337258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:33.336669    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 21:55:33.337258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:33.336736    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 21:55:33.386259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:33.386070    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:55:33.512261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:33.511708    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:33.537259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:33.537065    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:55:33.537475 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:33.537263    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 21:55:33.570258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:33.569903    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:33.570258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:33.570024    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:55:33.729292 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:33.729050    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:33.914260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:33.913726    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:55:33.914260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:33.913801    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:55:34.000318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:33.986823    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:34.098257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:34.097767    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:55:34.417334 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:34.404809    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 21:55:34.417334 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:34.404890    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 21:55:34.521825 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:34.521696    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:55:34.521825 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:34.521765    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 21:55:34.555767 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:34.555644    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:34.555767 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:34.555707    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:55:34.732775 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:34.732628    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:34.733486 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:34.733463    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:55:34.874088 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:34.874020    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" podStartSLOduration=4.873998029 podStartE2EDuration="4.873998029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:31.777283289 +0000 UTC m=+1274.449187207" watchObservedRunningTime="2026-10-30 21:55:34.873998029 +0000 UTC m=+1277.545901965"
Oct 30 21:55:34.895519 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:34.895480    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:55:34.895978 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:34.895948    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:55:35.183368 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:35.011286    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:35.183368 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:35.129726    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:55:35.183368 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:35.131377    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:55:35.183368 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:35.131458    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:55:35.337678 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:35.337638    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 21:55:35.337917 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:35.337891    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 21:55:35.511342 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:35.503183    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:55:35.647426 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:35.512117    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:35.647856 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:35.543856    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:55:35.647856 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:35.647803    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 21:55:35.647856 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:35.599675    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/health\": context deadline exceeded"
Oct 30 21:55:35.724269 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:35.699394    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:35.724269 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:35.699456    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:55:35.747267 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:35.736852    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:36.043496 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.043456    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:55:36.043759 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.043730    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:55:36.147490 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.147437    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:36.149039 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.149003    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:36.360327 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.356119    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 21:55:36.360639 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.360594    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 21:55:36.360849 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.360824    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/health\": context deadline exceeded"
Oct 30 21:55:36.533271 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.532391    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:55:36.533271 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.532464    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 21:55:36.580260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.569413    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:36.580260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.569483    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:55:36.660347 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.653923    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:55:36.660347 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.655273    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:55:36.671268 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.669363    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:36.673676 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.671637    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:55:36.758883 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:36.757342    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:55:37.146313 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.125474    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:55:37.146313 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.125561    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:55:37.149078 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.148177    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:37.150558 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.149382    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:37.382261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.373414    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 21:55:37.382261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.373489    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 21:55:37.437319 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.437287    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:55:37.592261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.558658    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:37.592261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.558737    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:55:37.704316 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.684953    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:55:37.704316 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.685114    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:37.824652 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.824611    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:55:37.828619 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.828578    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 21:55:37.911367 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.906411    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:55:37.911367 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.906480    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:55:37.924269 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:37.914582    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:38.058324 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:38.058046    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:55:38.114289 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:38.109131    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:55:38.211792 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:38.211617    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:55:38.211792 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:38.211691    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:55:38.680320 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:38.667533    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:55:38.680320 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:38.667603    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 21:55:38.680320 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:38.667713    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:38.680320 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:38.667738    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:55:38.920262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:38.895189    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:55:38.920262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:38.895293    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:55:39.413503 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:39.413454    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:39.414042 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:39.414007    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:39.540261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:39.539388    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:55:39.540261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:39.539460    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 21:55:39.556861 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:39.556822    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:39.557165 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:39.557124    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:55:39.720541 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:39.720438    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:39.755447 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:39.755387    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:39.904399 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:39.904353    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:39.932295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:39.926698    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:40.238259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:40.235354    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:40.314696 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:40.314649    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:40.318258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:40.317007    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:40.326613 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:40.326537    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 21:55:40.530892 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:40.530795    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Oct 30 21:55:40.532015 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:40.531982    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Oct 30 21:55:40.571260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:40.565463    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:40.571260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:40.565527    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:55:40.762175 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:40.761533    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:40.794742 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:40.794617    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:55:40.940507 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:40.940463    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:40.940991 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:40.940917    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:41.164260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:41.157368    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:55:41.181438 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:41.175398    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:55:41.232055 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:41.232009    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:55:41.244384 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:41.244332    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:41.248556 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:41.248520    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:55:41.559115 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:41.559020    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:55:41.559408 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:41.559370    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:55:41.787258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:41.783712    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:41.945407 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:41.945349    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:41.989465 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:41.989422    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:41.989815 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:41.989768    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:42.363260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:42.318404    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:42.459260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:42.445713    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:55:42.512849 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:42.443328    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:55:42.513499 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:42.405370    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:55:42.540921 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:42.540889    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:55:42.719702 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:42.700481    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:42.720005 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:42.719969    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:42.784872 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:42.784764    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:55:43.021270 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.014397    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:43.021270 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.014518    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" podUID="8310bcd1-2853-49f2-9618-0ff100c794bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:43.086320 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.085846    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:43.086320 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.085912    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:43.286261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.282722    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:43.318326 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.318291    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:43.337993 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.337897    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:43.353583 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.353553    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 21:55:43.501262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.489196    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:55:43.539487 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.539429    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:43.547381 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.547317    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:43.573576 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.551252    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:55:43.620999 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.620880    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:43.621295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.621260    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:43.812136 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.812098    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:43.812462 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.812418    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:43.814023 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:43.813992    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:44.379644 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.379609    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:44.383016 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.382989    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:44.409132 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.405988    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 21:55:44.504456 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.504413    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:55:44.551131 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.551087    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:44.603191 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.603111    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:44.603782 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.603642    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:55:44.626972 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.626944    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:55:44.645503 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.645016    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:55:44.669959 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.667356    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:44.669959 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.667421    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:44.761256 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.755766    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:44.762838 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.762803    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:44.766909 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.766861    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 21:55:44.816903 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.816862    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:44.828158 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.828124    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:55:44.838808 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.838779    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:55:44.964133 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:44.964100    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:55:45.006446 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:45.006409    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:55:48.186348 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:48.186313    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:48.362027 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:55:48.361991    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:55:49.789356 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:49.789319    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"]
Oct 30 21:55:49.794225 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:49.794203    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"
Oct 30 21:55:49.852696 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:49.851421    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:55:49.900294 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:49.900253    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 21:55:49.903898 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:49.903867    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"]
Oct 30 21:55:49.933063 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:49.933029    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f787aa0a-0804-4d85-9af8-0d50161693fb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bf234ca6-5978x\" (UID: \"f787aa0a-0804-4d85-9af8-0d50161693fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"
Oct 30 21:55:49.933222 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:49.933191    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f787aa0a-0804-4d85-9af8-0d50161693fb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bf234ca6-5978x\" (UID: \"f787aa0a-0804-4d85-9af8-0d50161693fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"
Oct 30 21:55:49.933283 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:49.933229    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lml8j\" (UniqueName: \"kubernetes.io/projected/f787aa0a-0804-4d85-9af8-0d50161693fb-kube-api-access-lml8j\") pod \"maas-api-revoke-sub-bf234ca6-5978x\" (UID: \"f787aa0a-0804-4d85-9af8-0d50161693fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"
Oct 30 21:55:50.035165 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:50.035128    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f787aa0a-0804-4d85-9af8-0d50161693fb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bf234ca6-5978x\" (UID: \"f787aa0a-0804-4d85-9af8-0d50161693fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"
Oct 30 21:55:50.035360 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:50.035215    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f787aa0a-0804-4d85-9af8-0d50161693fb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bf234ca6-5978x\" (UID: \"f787aa0a-0804-4d85-9af8-0d50161693fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"
Oct 30 21:55:50.035360 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:50.035280    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lml8j\" (UniqueName: \"kubernetes.io/projected/f787aa0a-0804-4d85-9af8-0d50161693fb-kube-api-access-lml8j\") pod \"maas-api-revoke-sub-bf234ca6-5978x\" (UID: \"f787aa0a-0804-4d85-9af8-0d50161693fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"
Oct 30 21:55:50.035831 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:50.035800    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f787aa0a-0804-4d85-9af8-0d50161693fb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bf234ca6-5978x\" (UID: \"f787aa0a-0804-4d85-9af8-0d50161693fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"
Oct 30 21:55:50.037604 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:50.037582    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f787aa0a-0804-4d85-9af8-0d50161693fb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bf234ca6-5978x\" (UID: \"f787aa0a-0804-4d85-9af8-0d50161693fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"
Oct 30 21:55:50.092306 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:50.092199    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lml8j\" (UniqueName: \"kubernetes.io/projected/f787aa0a-0804-4d85-9af8-0d50161693fb-kube-api-access-lml8j\") pod \"maas-api-revoke-sub-bf234ca6-5978x\" (UID: \"f787aa0a-0804-4d85-9af8-0d50161693fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"
Oct 30 21:55:50.110525 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:50.110484    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"
Oct 30 21:55:50.508716 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:50.508691    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"]
Oct 30 21:55:50.510467 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:55:50.510443    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf787aa0a_0804_4d85_9af8_0d50161693fb.slice/crio-1704084f9ed85404122ac9839398131454855978b5ddb0b16518a3eee677843d WatchSource:0}: Error finding container 1704084f9ed85404122ac9839398131454855978b5ddb0b16518a3eee677843d: Status 404 returned error can't find the container with id 1704084f9ed85404122ac9839398131454855978b5ddb0b16518a3eee677843d
Oct 30 21:55:50.834336 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:50.834300    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x" event={"ID":"f787aa0a-0804-4d85-9af8-0d50161693fb","Type":"ContainerStarted","Data":"1704084f9ed85404122ac9839398131454855978b5ddb0b16518a3eee677843d"}
Oct 30 21:55:51.842995 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:51.842962    5586 generic.go:356] "Generic (PLEG): container finished" podID="f787aa0a-0804-4d85-9af8-0d50161693fb" containerID="f35812722e70d61f5ed5c1de3508c8a08a0d500f7421db2edcb2540d48e747b2" exitCode=0
Oct 30 21:55:51.843481 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:51.843056    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x" event={"ID":"f787aa0a-0804-4d85-9af8-0d50161693fb","Type":"ContainerDied","Data":"f35812722e70d61f5ed5c1de3508c8a08a0d500f7421db2edcb2540d48e747b2"}
Oct 30 21:55:52.985774 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:52.985627    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:53.007826 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:53.007798    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"
Oct 30 21:55:53.080066 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:53.080024    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f787aa0a-0804-4d85-9af8-0d50161693fb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f787aa0a-0804-4d85-9af8-0d50161693fb-maas-api-xks-ca\") pod \"f787aa0a-0804-4d85-9af8-0d50161693fb\" (UID: \"f787aa0a-0804-4d85-9af8-0d50161693fb\") "
Oct 30 21:55:53.080324 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:53.080224    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f787aa0a-0804-4d85-9af8-0d50161693fb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f787aa0a-0804-4d85-9af8-0d50161693fb-maas-api-openshift-ca\") pod \"f787aa0a-0804-4d85-9af8-0d50161693fb\" (UID: \"f787aa0a-0804-4d85-9af8-0d50161693fb\") "
Oct 30 21:55:53.080405 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:53.080384    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f787aa0a-0804-4d85-9af8-0d50161693fb-kube-api-access-lml8j\" (UniqueName: \"kubernetes.io/projected/f787aa0a-0804-4d85-9af8-0d50161693fb-kube-api-access-lml8j\") pod \"f787aa0a-0804-4d85-9af8-0d50161693fb\" (UID: \"f787aa0a-0804-4d85-9af8-0d50161693fb\") "
Oct 30 21:55:53.081287 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:53.081248    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f787aa0a-0804-4d85-9af8-0d50161693fb-maas-api-openshift-ca" pod "f787aa0a-0804-4d85-9af8-0d50161693fb" (UID: "f787aa0a-0804-4d85-9af8-0d50161693fb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:53.083110 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:53.083080    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f787aa0a-0804-4d85-9af8-0d50161693fb-maas-api-xks-ca" pod "f787aa0a-0804-4d85-9af8-0d50161693fb" (UID: "f787aa0a-0804-4d85-9af8-0d50161693fb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:53.083487 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:53.083469    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f787aa0a-0804-4d85-9af8-0d50161693fb-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:53.083571 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:53.083492    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f787aa0a-0804-4d85-9af8-0d50161693fb-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:53.084383 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:53.084363    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f787aa0a-0804-4d85-9af8-0d50161693fb-kube-api-access-lml8j" pod "f787aa0a-0804-4d85-9af8-0d50161693fb" (UID: "f787aa0a-0804-4d85-9af8-0d50161693fb"). InnerVolumeSpecName "kube-api-access-lml8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:53.185012 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:53.184976    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lml8j\" (UniqueName: \"kubernetes.io/projected/f787aa0a-0804-4d85-9af8-0d50161693fb-kube-api-access-lml8j\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:53.858592 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:53.858501    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"
Oct 30 21:55:53.858592 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:53.858527    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x" event={"ID":"f787aa0a-0804-4d85-9af8-0d50161693fb","Type":"ContainerDied","Data":"1704084f9ed85404122ac9839398131454855978b5ddb0b16518a3eee677843d"}
Oct 30 21:55:53.858592 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:53.858589    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1704084f9ed85404122ac9839398131454855978b5ddb0b16518a3eee677843d"
Oct 30 21:55:57.981672 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:55:57.981515    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:00.116207 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.116173    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"]
Oct 30 21:56:00.117036 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.117013    5586 cpu_manager.go:414] "containerMap: removing container" podUID="f787aa0a-0804-4d85-9af8-0d50161693fb" containerName="revoke-keys"
Oct 30 21:56:00.117143 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.117038    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f787aa0a-0804-4d85-9af8-0d50161693fb" containerName="revoke-keys"
Oct 30 21:56:00.117317 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.117291    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="f787aa0a-0804-4d85-9af8-0d50161693fb" containerName="revoke-keys"
Oct 30 21:56:00.121021 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.120999    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.187688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.187644    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.187889 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.187701    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.187889 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.187824    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.187889 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.187878    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.188087 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.187974    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a027617-89f1-4e5e-84f0-4bc510c9b357-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.188087 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.188002    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.188195 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.188177    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cjc2\" (UniqueName: \"kubernetes.io/projected/1a027617-89f1-4e5e-84f0-4bc510c9b357-kube-api-access-2cjc2\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.188268 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.188209    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.188341 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.188292    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.229915 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.229842    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"]
Oct 30 21:56:00.289692 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.289651    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cjc2\" (UniqueName: \"kubernetes.io/projected/1a027617-89f1-4e5e-84f0-4bc510c9b357-kube-api-access-2cjc2\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.289870 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.289698    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.289870 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.289762    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.289870 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.289855    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.290011 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.289981    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.290087 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.290073    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.290145 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.290120    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.290198 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.290165    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-workload-certs\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.290267 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.290201    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-data\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.290267 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.290219    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a027617-89f1-4e5e-84f0-4bc510c9b357-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.290362 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.290338    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.290439 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.290418    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-workload-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.290706 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.290676    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-credential-socket\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.290948 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.290930    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a027617-89f1-4e5e-84f0-4bc510c9b357-istiod-ca-cert\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.292279 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.292260    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-envoy\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.292599 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.292583    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-podinfo\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.351751 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.351720    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-token\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.351892 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.351726    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cjc2\" (UniqueName: \"kubernetes.io/projected/1a027617-89f1-4e5e-84f0-4bc510c9b357-kube-api-access-2cjc2\") pod \"e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") " pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.434824 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.434788    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:00.740047 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.739962    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"]
Oct 30 21:56:00.745447 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.745422    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.798493 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.798457    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.798699 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.798505    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.798699 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.798541    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.798699 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.798628    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.798849 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.798706    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.798905 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.798878    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjhtm\" (UniqueName: \"kubernetes.io/projected/d88dfc7e-36a5-4dd1-9814-2b966c89b085-kube-api-access-gjhtm\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.799052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.799032    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.799144 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.799130    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.799202 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.799154    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.817957 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.817923    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"]
Oct 30 21:56:00.900325 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.900289    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.900512 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.900369    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.900512 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.900399    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.900512 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.900423    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjhtm\" (UniqueName: \"kubernetes.io/projected/d88dfc7e-36a5-4dd1-9814-2b966c89b085-kube-api-access-gjhtm\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.900512 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.900483    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.900660 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.900552    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.900660 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.900578    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.900752 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.900699    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.900752 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.900736    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.900847 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.900751    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-workload-certs\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.901115 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.901055    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-data\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.901333 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.901307    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istiod-ca-cert\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.901447 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.901429    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-workload-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.901490 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.901463    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-credential-socket\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.903866 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.903833    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-podinfo\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.904188 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.904169    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-envoy\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.907541 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.907516    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"]
Oct 30 21:56:00.907881 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:56:00.907861    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a027617_89f1_4e5e_84f0_4bc510c9b357.slice/crio-35c44cc302927b5483389545e6475d33ec820bf16a40ff9a7e98b9bccd08cd38 WatchSource:0}: Error finding container 35c44cc302927b5483389545e6475d33ec820bf16a40ff9a7e98b9bccd08cd38: Status 404 returned error can't find the container with id 35c44cc302927b5483389545e6475d33ec820bf16a40ff9a7e98b9bccd08cd38
Oct 30 21:56:00.910577 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.910544    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:00.910655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.910609    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:00.910655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.910640    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:00.917708 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.917678    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr" event={"ID":"1a027617-89f1-4e5e-84f0-4bc510c9b357","Type":"ContainerStarted","Data":"35c44cc302927b5483389545e6475d33ec820bf16a40ff9a7e98b9bccd08cd38"}
Oct 30 21:56:00.959757 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.959726    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-token\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:00.960125 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:00.960105    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjhtm\" (UniqueName: \"kubernetes.io/projected/d88dfc7e-36a5-4dd1-9814-2b966c89b085-kube-api-access-gjhtm\") pod \"e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") " pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:01.060951 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:01.060854    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:01.561964 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:01.561932    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"]
Oct 30 21:56:01.563114 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:56:01.563087    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd88dfc7e_36a5_4dd1_9814_2b966c89b085.slice/crio-eb0a29815889f8fc135577f2895a55e64b11838e6895a46876212323a35b708c WatchSource:0}: Error finding container eb0a29815889f8fc135577f2895a55e64b11838e6895a46876212323a35b708c: Status 404 returned error can't find the container with id eb0a29815889f8fc135577f2895a55e64b11838e6895a46876212323a35b708c
Oct 30 21:56:01.566367 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:01.566333    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:01.566455 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:01.566399    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:01.566455 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:01.566427    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:01.953458 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:01.953403    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" event={"ID":"d88dfc7e-36a5-4dd1-9814-2b966c89b085","Type":"ContainerStarted","Data":"ec47de0245618ac8759325c15d25ca595a629c69f9b980867c547d94346b832b"}
Oct 30 21:56:01.953664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:01.953478    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" event={"ID":"d88dfc7e-36a5-4dd1-9814-2b966c89b085","Type":"ContainerStarted","Data":"eb0a29815889f8fc135577f2895a55e64b11838e6895a46876212323a35b708c"}
Oct 30 21:56:01.955410 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:01.955378    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr" event={"ID":"1a027617-89f1-4e5e-84f0-4bc510c9b357","Type":"ContainerStarted","Data":"b0626bfc4b5fb425254a26a649f25184b685cab13989321c89b1cea86c3adfab"}
Oct 30 21:56:02.061503 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:02.061444    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:02.062833 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:02.062807    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:56:02.062969 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:02.062863    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" podUID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 21:56:02.074273 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:02.074208    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" podStartSLOduration=2.074191746 podStartE2EDuration="2.074191746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:02.072315168 +0000 UTC m=+1304.744219103" watchObservedRunningTime="2026-10-30 21:56:02.074191746 +0000 UTC m=+1304.746095680"
Oct 30 21:56:02.203620 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:02.203509    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr" podStartSLOduration=2.203494396 podStartE2EDuration="2.203494396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:02.199438996 +0000 UTC m=+1304.871342928" watchObservedRunningTime="2026-10-30 21:56:02.203494396 +0000 UTC m=+1304.875398330"
Oct 30 21:56:02.461321 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:02.450007    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:02.474941 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:02.474413    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 21:56:02.474941 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:02.474480    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr" podUID="1a027617-89f1-4e5e-84f0-4bc510c9b357" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 21:56:03.061729 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:03.061688    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:56:03.062325 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:03.062291    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" podUID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 21:56:03.442949 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:03.442912    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:03.464895 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:03.464859    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 21:56:03.465179 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:03.465147    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr" podUID="1a027617-89f1-4e5e-84f0-4bc510c9b357" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 21:56:04.072344 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:04.068070    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:56:04.072344 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:04.068132    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" podUID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 21:56:04.441259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:04.437578    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 21:56:04.441259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:04.437640    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr" podUID="1a027617-89f1-4e5e-84f0-4bc510c9b357" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 21:56:05.072117 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:05.071418    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:56:05.072117 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:05.071479    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" podUID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 21:56:05.480460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:05.480414    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 21:56:05.480966 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:05.480481    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr" podUID="1a027617-89f1-4e5e-84f0-4bc510c9b357" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 21:56:06.076286 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.075651    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:56:06.076286 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.075716    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" podUID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 21:56:06.377303 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.376079    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"]
Oct 30 21:56:06.380187 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.380167    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"
Oct 30 21:56:06.442260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.439547    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 21:56:06.442260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.441525    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fszd6\" (UniqueName: \"kubernetes.io/projected/d3235316-025d-4a9a-9315-7a80e65dcb66-kube-api-access-fszd6\") pod \"maas-api-revoke-sub-18505428-57hqw\" (UID: \"d3235316-025d-4a9a-9315-7a80e65dcb66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"
Oct 30 21:56:06.442260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.441665    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3235316-025d-4a9a-9315-7a80e65dcb66-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-18505428-57hqw\" (UID: \"d3235316-025d-4a9a-9315-7a80e65dcb66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"
Oct 30 21:56:06.442260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.441876    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3235316-025d-4a9a-9315-7a80e65dcb66-maas-api-xks-ca\") pod \"maas-api-revoke-sub-18505428-57hqw\" (UID: \"d3235316-025d-4a9a-9315-7a80e65dcb66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"
Oct 30 21:56:06.446267 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.443800    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 21:56:06.446267 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.443854    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr" podUID="1a027617-89f1-4e5e-84f0-4bc510c9b357" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 21:56:06.493220 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.493189    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"]
Oct 30 21:56:06.575217 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.575172    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3235316-025d-4a9a-9315-7a80e65dcb66-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-18505428-57hqw\" (UID: \"d3235316-025d-4a9a-9315-7a80e65dcb66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"
Oct 30 21:56:06.576087 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.576059    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3235316-025d-4a9a-9315-7a80e65dcb66-maas-api-xks-ca\") pod \"maas-api-revoke-sub-18505428-57hqw\" (UID: \"d3235316-025d-4a9a-9315-7a80e65dcb66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"
Oct 30 21:56:06.576534 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.576509    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fszd6\" (UniqueName: \"kubernetes.io/projected/d3235316-025d-4a9a-9315-7a80e65dcb66-kube-api-access-fszd6\") pod \"maas-api-revoke-sub-18505428-57hqw\" (UID: \"d3235316-025d-4a9a-9315-7a80e65dcb66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"
Oct 30 21:56:06.577125 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.577080    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:56:06.578153 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.578133    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3235316-025d-4a9a-9315-7a80e65dcb66-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-18505428-57hqw\" (UID: \"d3235316-025d-4a9a-9315-7a80e65dcb66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"
Oct 30 21:56:06.590140 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.590117    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3235316-025d-4a9a-9315-7a80e65dcb66-maas-api-xks-ca\") pod \"maas-api-revoke-sub-18505428-57hqw\" (UID: \"d3235316-025d-4a9a-9315-7a80e65dcb66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"
Oct 30 21:56:06.675308 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.673996    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fszd6\" (UniqueName: \"kubernetes.io/projected/d3235316-025d-4a9a-9315-7a80e65dcb66-kube-api-access-fszd6\") pod \"maas-api-revoke-sub-18505428-57hqw\" (UID: \"d3235316-025d-4a9a-9315-7a80e65dcb66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"
Oct 30 21:56:06.694274 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:06.694216    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"
Oct 30 21:56:07.062314 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:07.062201    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:56:07.062314 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:07.062283    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" podUID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 21:56:07.157589 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:07.157479    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"]
Oct 30 21:56:07.161165 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:56:07.161137    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3235316_025d_4a9a_9315_7a80e65dcb66.slice/crio-2086a58667bbb01faa6cdcd88e9026b1f2587e84f47b0948fcb14d58489c3a41 WatchSource:0}: Error finding container 2086a58667bbb01faa6cdcd88e9026b1f2587e84f47b0948fcb14d58489c3a41: Status 404 returned error can't find the container with id 2086a58667bbb01faa6cdcd88e9026b1f2587e84f47b0948fcb14d58489c3a41
Oct 30 21:56:07.444357 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:07.444323    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused" start-of-body=
Oct 30 21:56:07.444637 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:07.444598    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr" podUID="1a027617-89f1-4e5e-84f0-4bc510c9b357" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": dial tcp 10.134.0.87:15021: connect: connection refused"
Oct 30 21:56:08.023269 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:08.023198    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw" event={"ID":"d3235316-025d-4a9a-9315-7a80e65dcb66","Type":"ContainerStarted","Data":"bbb49e79897a75ba7fca41e78ee5bd64caf3be2fc0a092da281e71e1de0b13bb"}
Oct 30 21:56:08.023895 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:08.023855    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw" event={"ID":"d3235316-025d-4a9a-9315-7a80e65dcb66","Type":"ContainerStarted","Data":"2086a58667bbb01faa6cdcd88e9026b1f2587e84f47b0948fcb14d58489c3a41"}
Oct 30 21:56:08.062281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:08.062218    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Oct 30 21:56:08.062460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:08.062321    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" podUID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Oct 30 21:56:08.187535 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:08.187470    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw" podStartSLOduration=2.187448442 podStartE2EDuration="2.187448442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:08.145829175 +0000 UTC m=+1310.817733131" watchObservedRunningTime="2026-10-30 21:56:08.187448442 +0000 UTC m=+1310.859352384"
Oct 30 21:56:08.485664 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:08.485627    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:08.487363 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:08.487336    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:08.488274 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:08.488251    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 21:56:08.811596 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:08.811492    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:09.049290 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:09.049208    5586 generic.go:356] "Generic (PLEG): container finished" podID="d3235316-025d-4a9a-9315-7a80e65dcb66" containerID="bbb49e79897a75ba7fca41e78ee5bd64caf3be2fc0a092da281e71e1de0b13bb" exitCode=0
Oct 30 21:56:09.050216 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:09.049648    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw" event={"ID":"d3235316-025d-4a9a-9315-7a80e65dcb66","Type":"ContainerDied","Data":"bbb49e79897a75ba7fca41e78ee5bd64caf3be2fc0a092da281e71e1de0b13bb"}
Oct 30 21:56:09.309258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:09.308401    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:09.310267 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:09.309827    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:09.311576 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:09.311545    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 21:56:10.229795 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:10.229770    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"
Oct 30 21:56:10.343070 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:10.342987    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3235316-025d-4a9a-9315-7a80e65dcb66-kube-api-access-fszd6\" (UniqueName: \"kubernetes.io/projected/d3235316-025d-4a9a-9315-7a80e65dcb66-kube-api-access-fszd6\") pod \"d3235316-025d-4a9a-9315-7a80e65dcb66\" (UID: \"d3235316-025d-4a9a-9315-7a80e65dcb66\") "
Oct 30 21:56:10.343221 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:10.343201    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3235316-025d-4a9a-9315-7a80e65dcb66-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3235316-025d-4a9a-9315-7a80e65dcb66-maas-api-openshift-ca\") pod \"d3235316-025d-4a9a-9315-7a80e65dcb66\" (UID: \"d3235316-025d-4a9a-9315-7a80e65dcb66\") "
Oct 30 21:56:10.343302 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:10.343277    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3235316-025d-4a9a-9315-7a80e65dcb66-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3235316-025d-4a9a-9315-7a80e65dcb66-maas-api-xks-ca\") pod \"d3235316-025d-4a9a-9315-7a80e65dcb66\" (UID: \"d3235316-025d-4a9a-9315-7a80e65dcb66\") "
Oct 30 21:56:10.343613 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:10.343583    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3235316-025d-4a9a-9315-7a80e65dcb66-maas-api-openshift-ca" pod "d3235316-025d-4a9a-9315-7a80e65dcb66" (UID: "d3235316-025d-4a9a-9315-7a80e65dcb66"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:10.343875 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:10.343857    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3235316-025d-4a9a-9315-7a80e65dcb66-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:10.346229 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:10.346203    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3235316-025d-4a9a-9315-7a80e65dcb66-maas-api-xks-ca" pod "d3235316-025d-4a9a-9315-7a80e65dcb66" (UID: "d3235316-025d-4a9a-9315-7a80e65dcb66"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:10.346336 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:10.346307    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3235316-025d-4a9a-9315-7a80e65dcb66-kube-api-access-fszd6" pod "d3235316-025d-4a9a-9315-7a80e65dcb66" (UID: "d3235316-025d-4a9a-9315-7a80e65dcb66"). InnerVolumeSpecName "kube-api-access-fszd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:10.445652 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:10.445618    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3235316-025d-4a9a-9315-7a80e65dcb66-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:10.445652 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:10.445647    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fszd6\" (UniqueName: \"kubernetes.io/projected/d3235316-025d-4a9a-9315-7a80e65dcb66-kube-api-access-fszd6\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:11.066955 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:11.066917    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"
Oct 30 21:56:11.067160 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:11.066953    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw" event={"ID":"d3235316-025d-4a9a-9315-7a80e65dcb66","Type":"ContainerDied","Data":"2086a58667bbb01faa6cdcd88e9026b1f2587e84f47b0948fcb14d58489c3a41"}
Oct 30 21:56:11.067160 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:11.067004    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2086a58667bbb01faa6cdcd88e9026b1f2587e84f47b0948fcb14d58489c3a41"
Oct 30 21:56:13.638922 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:13.638879    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:17.571339 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:17.571293    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:56:17.637079 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:17.637031    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:17.710952 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:17.710886    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:17.793880 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:17.793840    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:17.851422 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:17.843634    5586 scope.go:122] "RemoveContainer" containerID="1f4abcb1cd2ecfb2855a12e74c4ef14ea183c67d308f725820097a8b9239924d"
Oct 30 21:56:18.253001 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:18.252942    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:18.718260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:18.715659    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:18.718260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:18.716540    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:18.718260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:18.717669    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:56:18.858175 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:18.736216    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:56:19.001814 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:19.001776    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:56:19.022266 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:18.790022    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:56:19.073266 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:18.803137    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:19.081145 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:19.081110    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:56:19.122389 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:19.122356    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:56:19.326619 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:19.326521    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"]
Oct 30 21:56:19.404261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:19.379586    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-784d86fa-pzpsc"]
Oct 30 21:56:19.838260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:19.702137    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50340bb1-68cf-4f15-bd32-5ad69697188c" path="/var/lib/kubelet/pods/50340bb1-68cf-4f15-bd32-5ad69697188c/volumes"
Oct 30 21:56:19.853281 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:19.817765    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:56:19.853628 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:19.817802    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:56:19.885353 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:19.842365    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:20.032487 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:19.885896    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:56:20.045399 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:20.001438    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:56:20.091317 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:20.069460    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:20.127258 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:56:20.072567    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:56:20.127258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:20.073692    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:56:20.520139 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:20.520097    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:20.522393 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:20.522337    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:20.857954 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:20.857826    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:56:20.884740 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:20.884694    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:21.064250 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:21.064193    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:56:21.150349 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:21.125052    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:21.150349 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:21.127792    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:56:21.871687 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:21.870761    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:56:21.886438 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:21.886392    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:21.939246 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:21.939133    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:22.187691 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:22.187645    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:22.188765 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:22.188732    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:56:22.260769 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:22.220604    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:56:22.510366 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:22.510098    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:56:22.514791 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:22.514527    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:56:22.526510 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:22.526472    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:56:22.579889 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:22.579853    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:56:22.655505 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:22.654519    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:56:23.071285 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:23.071246    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:23.234813 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:23.234772    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:25.264169 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.264136    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"]
Oct 30 21:56:25.264848 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.264834    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d3235316-025d-4a9a-9315-7a80e65dcb66" containerName="revoke-keys"
Oct 30 21:56:25.264896 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.264850    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3235316-025d-4a9a-9315-7a80e65dcb66" containerName="revoke-keys"
Oct 30 21:56:25.264986 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.264977    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3235316-025d-4a9a-9315-7a80e65dcb66" containerName="revoke-keys"
Oct 30 21:56:25.268562 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.268545    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"
Oct 30 21:56:25.320671 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.320635    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/661fbff3-6968-41c6-b8d1-ab9f2b22820a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ec238e1-gj8bb\" (UID: \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"
Oct 30 21:56:25.320845 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.320820    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/661fbff3-6968-41c6-b8d1-ab9f2b22820a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ec238e1-gj8bb\" (UID: \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"
Oct 30 21:56:25.320932 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.320916    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vjt9\" (UniqueName: \"kubernetes.io/projected/661fbff3-6968-41c6-b8d1-ab9f2b22820a-kube-api-access-8vjt9\") pod \"maas-api-revoke-sub-9ec238e1-gj8bb\" (UID: \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"
Oct 30 21:56:25.329514 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.329482    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"]
Oct 30 21:56:25.373847 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.373816    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:56:25.374051 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.373946    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 21:56:25.422071 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.422037    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/661fbff3-6968-41c6-b8d1-ab9f2b22820a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ec238e1-gj8bb\" (UID: \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"
Oct 30 21:56:25.422288 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.422137    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vjt9\" (UniqueName: \"kubernetes.io/projected/661fbff3-6968-41c6-b8d1-ab9f2b22820a-kube-api-access-8vjt9\") pod \"maas-api-revoke-sub-9ec238e1-gj8bb\" (UID: \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"
Oct 30 21:56:25.422288 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.422264    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/661fbff3-6968-41c6-b8d1-ab9f2b22820a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ec238e1-gj8bb\" (UID: \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"
Oct 30 21:56:25.422751 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.422726    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/661fbff3-6968-41c6-b8d1-ab9f2b22820a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ec238e1-gj8bb\" (UID: \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"
Oct 30 21:56:25.424728 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.424707    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/661fbff3-6968-41c6-b8d1-ab9f2b22820a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ec238e1-gj8bb\" (UID: \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"
Oct 30 21:56:25.480490 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.480451    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vjt9\" (UniqueName: \"kubernetes.io/projected/661fbff3-6968-41c6-b8d1-ab9f2b22820a-kube-api-access-8vjt9\") pod \"maas-api-revoke-sub-9ec238e1-gj8bb\" (UID: \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"
Oct 30 21:56:25.582899 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.582807    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"
Oct 30 21:56:25.974146 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:25.974111    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"]
Oct 30 21:56:25.974386 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:56:25.974361    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod661fbff3_6968_41c6_b8d1_ab9f2b22820a.slice/crio-cd6a6a407bf7e1df51fe9f8c08f111f5071f8e3cf31b22b6ce86194dd83fdba9 WatchSource:0}: Error finding container cd6a6a407bf7e1df51fe9f8c08f111f5071f8e3cf31b22b6ce86194dd83fdba9: Status 404 returned error can't find the container with id cd6a6a407bf7e1df51fe9f8c08f111f5071f8e3cf31b22b6ce86194dd83fdba9
Oct 30 21:56:26.295135 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:26.295099    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb" event={"ID":"661fbff3-6968-41c6-b8d1-ab9f2b22820a","Type":"ContainerStarted","Data":"fdc1c48895e6b8d6dd9928e8fc970813444244af128f983963d00416c0c67e9f"}
Oct 30 21:56:26.296029 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:26.295159    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb" event={"ID":"661fbff3-6968-41c6-b8d1-ab9f2b22820a","Type":"ContainerStarted","Data":"cd6a6a407bf7e1df51fe9f8c08f111f5071f8e3cf31b22b6ce86194dd83fdba9"}
Oct 30 21:56:26.412911 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:26.412818    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb" podStartSLOduration=1.41279786 podStartE2EDuration="1.41279786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:26.409379406 +0000 UTC m=+1329.081283342" watchObservedRunningTime="2026-10-30 21:56:26.41279786 +0000 UTC m=+1329.084701803"
Oct 30 21:56:26.429026 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:26.428062    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:27.304764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:27.304725    5586 generic.go:356] "Generic (PLEG): container finished" podID="661fbff3-6968-41c6-b8d1-ab9f2b22820a" containerID="fdc1c48895e6b8d6dd9928e8fc970813444244af128f983963d00416c0c67e9f" exitCode=0
Oct 30 21:56:27.305265 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:27.304802    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb" event={"ID":"661fbff3-6968-41c6-b8d1-ab9f2b22820a","Type":"ContainerDied","Data":"fdc1c48895e6b8d6dd9928e8fc970813444244af128f983963d00416c0c67e9f"}
Oct 30 21:56:28.483045 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:28.483018    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"
Oct 30 21:56:28.568188 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:28.568103    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/661fbff3-6968-41c6-b8d1-ab9f2b22820a-kube-api-access-8vjt9\" (UniqueName: \"kubernetes.io/projected/661fbff3-6968-41c6-b8d1-ab9f2b22820a-kube-api-access-8vjt9\") pod \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\" (UID: \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\") "
Oct 30 21:56:28.568367 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:28.568334    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/661fbff3-6968-41c6-b8d1-ab9f2b22820a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/661fbff3-6968-41c6-b8d1-ab9f2b22820a-maas-api-openshift-ca\") pod \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\" (UID: \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\") "
Oct 30 21:56:28.568473 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:28.568460    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/661fbff3-6968-41c6-b8d1-ab9f2b22820a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/661fbff3-6968-41c6-b8d1-ab9f2b22820a-maas-api-xks-ca\") pod \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\" (UID: \"661fbff3-6968-41c6-b8d1-ab9f2b22820a\") "
Oct 30 21:56:28.568695 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:28.568669    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/661fbff3-6968-41c6-b8d1-ab9f2b22820a-maas-api-openshift-ca" pod "661fbff3-6968-41c6-b8d1-ab9f2b22820a" (UID: "661fbff3-6968-41c6-b8d1-ab9f2b22820a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:28.568901 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:28.568885    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/661fbff3-6968-41c6-b8d1-ab9f2b22820a-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:28.570225 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:28.570203    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/661fbff3-6968-41c6-b8d1-ab9f2b22820a-kube-api-access-8vjt9" pod "661fbff3-6968-41c6-b8d1-ab9f2b22820a" (UID: "661fbff3-6968-41c6-b8d1-ab9f2b22820a"). InnerVolumeSpecName "kube-api-access-8vjt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:28.570481 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:28.570461    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/661fbff3-6968-41c6-b8d1-ab9f2b22820a-maas-api-xks-ca" pod "661fbff3-6968-41c6-b8d1-ab9f2b22820a" (UID: "661fbff3-6968-41c6-b8d1-ab9f2b22820a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:28.670524 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:28.670489    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/661fbff3-6968-41c6-b8d1-ab9f2b22820a-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:28.670524 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:28.670523    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vjt9\" (UniqueName: \"kubernetes.io/projected/661fbff3-6968-41c6-b8d1-ab9f2b22820a-kube-api-access-8vjt9\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:29.321270 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:29.321225    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"
Oct 30 21:56:29.321450 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:29.321267    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb" event={"ID":"661fbff3-6968-41c6-b8d1-ab9f2b22820a","Type":"ContainerDied","Data":"cd6a6a407bf7e1df51fe9f8c08f111f5071f8e3cf31b22b6ce86194dd83fdba9"}
Oct 30 21:56:29.321450 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:29.321330    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd6a6a407bf7e1df51fe9f8c08f111f5071f8e3cf31b22b6ce86194dd83fdba9"
Oct 30 21:56:31.336964 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:31.336925    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:32.336842 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.336801    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"]
Oct 30 21:56:32.337895 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.337856    5586 cpu_manager.go:414] "containerMap: removing container" podUID="661fbff3-6968-41c6-b8d1-ab9f2b22820a" containerName="revoke-keys"
Oct 30 21:56:32.337895 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.337888    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="661fbff3-6968-41c6-b8d1-ab9f2b22820a" containerName="revoke-keys"
Oct 30 21:56:32.338408 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.338113    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="661fbff3-6968-41c6-b8d1-ab9f2b22820a" containerName="revoke-keys"
Oct 30 21:56:32.342457 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.342436    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"
Oct 30 21:56:32.397604 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.397573    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"]
Oct 30 21:56:32.431667 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.431620    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9110ee28-abf0-46dd-a2e7-deec072b5eb6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bbcf2648-9v5bm\" (UID: \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"
Oct 30 21:56:32.431822 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.431800    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9110ee28-abf0-46dd-a2e7-deec072b5eb6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bbcf2648-9v5bm\" (UID: \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"
Oct 30 21:56:32.431922 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.431905    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chsv4\" (UniqueName: \"kubernetes.io/projected/9110ee28-abf0-46dd-a2e7-deec072b5eb6-kube-api-access-chsv4\") pod \"maas-api-revoke-sub-bbcf2648-9v5bm\" (UID: \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"
Oct 30 21:56:32.441600 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.441561    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 21:56:32.441600 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.441590    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:56:32.533748 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.533704    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9110ee28-abf0-46dd-a2e7-deec072b5eb6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bbcf2648-9v5bm\" (UID: \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"
Oct 30 21:56:32.533926 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.533796    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chsv4\" (UniqueName: \"kubernetes.io/projected/9110ee28-abf0-46dd-a2e7-deec072b5eb6-kube-api-access-chsv4\") pod \"maas-api-revoke-sub-bbcf2648-9v5bm\" (UID: \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"
Oct 30 21:56:32.534091 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.534072    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9110ee28-abf0-46dd-a2e7-deec072b5eb6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bbcf2648-9v5bm\" (UID: \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"
Oct 30 21:56:32.534680 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.534645    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9110ee28-abf0-46dd-a2e7-deec072b5eb6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bbcf2648-9v5bm\" (UID: \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"
Oct 30 21:56:32.536152 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.536135    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9110ee28-abf0-46dd-a2e7-deec072b5eb6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bbcf2648-9v5bm\" (UID: \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"
Oct 30 21:56:32.590748 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.590653    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chsv4\" (UniqueName: \"kubernetes.io/projected/9110ee28-abf0-46dd-a2e7-deec072b5eb6-kube-api-access-chsv4\") pod \"maas-api-revoke-sub-bbcf2648-9v5bm\" (UID: \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"
Oct 30 21:56:32.655890 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:32.655858    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"
Oct 30 21:56:33.044741 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.044710    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"]
Oct 30 21:56:33.045211 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:56:33.045189    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9110ee28_abf0_46dd_a2e7_deec072b5eb6.slice/crio-66d5e5470621d58ecdee59a1710777343926abc743a7cafaaaa2937ad0ce476e WatchSource:0}: Error finding container 66d5e5470621d58ecdee59a1710777343926abc743a7cafaaaa2937ad0ce476e: Status 404 returned error can't find the container with id 66d5e5470621d58ecdee59a1710777343926abc743a7cafaaaa2937ad0ce476e
Oct 30 21:56:33.320534 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.320453    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"]
Oct 30 21:56:33.326156 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.326135    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"
Oct 30 21:56:33.354296 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.354262    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm" event={"ID":"9110ee28-abf0-46dd-a2e7-deec072b5eb6","Type":"ContainerStarted","Data":"2de51ed0df230931e42cb701379e4adc86a0ef0d78971eb69db2179440dab486"}
Oct 30 21:56:33.354686 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.354317    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm" event={"ID":"9110ee28-abf0-46dd-a2e7-deec072b5eb6","Type":"ContainerStarted","Data":"66d5e5470621d58ecdee59a1710777343926abc743a7cafaaaa2937ad0ce476e"}
Oct 30 21:56:33.381740 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.381705    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"]
Oct 30 21:56:33.453764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.453720    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b4b07881-jbhhs\" (UID: \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"
Oct 30 21:56:33.454014 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.453795    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b4b07881-jbhhs\" (UID: \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"
Oct 30 21:56:33.454014 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.453900    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj7dt\" (UniqueName: \"kubernetes.io/projected/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-kube-api-access-zj7dt\") pod \"maas-api-revoke-sub-b4b07881-jbhhs\" (UID: \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"
Oct 30 21:56:33.554084 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.554028    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm" podStartSLOduration=1.5540129980000001 podStartE2EDuration="1.554012998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:33.549897426 +0000 UTC m=+1336.221801362" watchObservedRunningTime="2026-10-30 21:56:33.554012998 +0000 UTC m=+1336.225916936"
Oct 30 21:56:33.556049 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.556019    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b4b07881-jbhhs\" (UID: \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"
Oct 30 21:56:33.556166 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.556071    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b4b07881-jbhhs\" (UID: \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"
Oct 30 21:56:33.556209 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.556177    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj7dt\" (UniqueName: \"kubernetes.io/projected/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-kube-api-access-zj7dt\") pod \"maas-api-revoke-sub-b4b07881-jbhhs\" (UID: \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"
Oct 30 21:56:33.556746 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.556720    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b4b07881-jbhhs\" (UID: \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"
Oct 30 21:56:33.558566 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.558538    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b4b07881-jbhhs\" (UID: \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"
Oct 30 21:56:33.613562 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.613482    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj7dt\" (UniqueName: \"kubernetes.io/projected/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-kube-api-access-zj7dt\") pod \"maas-api-revoke-sub-b4b07881-jbhhs\" (UID: \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"
Oct 30 21:56:33.642658 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:33.642604    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"
Oct 30 21:56:34.025530 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:34.025496    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"]
Oct 30 21:56:34.026715 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:56:34.026689    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode54cc187_16f5_4efa_8d02_1e91b0a99d1c.slice/crio-490f5366148a2493120c0beb8f6fd6e23dee0b72eccb79c3f5b520995835caa2 WatchSource:0}: Error finding container 490f5366148a2493120c0beb8f6fd6e23dee0b72eccb79c3f5b520995835caa2: Status 404 returned error can't find the container with id 490f5366148a2493120c0beb8f6fd6e23dee0b72eccb79c3f5b520995835caa2
Oct 30 21:56:34.363901 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:34.363852    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs" event={"ID":"e54cc187-16f5-4efa-8d02-1e91b0a99d1c","Type":"ContainerStarted","Data":"dfb262670e3e32ddeaba9413a49d1c06009f48af6c3bf22568764b1db95e4109"}
Oct 30 21:56:34.364438 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:34.363920    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs" event={"ID":"e54cc187-16f5-4efa-8d02-1e91b0a99d1c","Type":"ContainerStarted","Data":"490f5366148a2493120c0beb8f6fd6e23dee0b72eccb79c3f5b520995835caa2"}
Oct 30 21:56:34.365615 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:34.365592    5586 generic.go:356] "Generic (PLEG): container finished" podID="9110ee28-abf0-46dd-a2e7-deec072b5eb6" containerID="2de51ed0df230931e42cb701379e4adc86a0ef0d78971eb69db2179440dab486" exitCode=0
Oct 30 21:56:34.365698 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:34.365668    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm" event={"ID":"9110ee28-abf0-46dd-a2e7-deec072b5eb6","Type":"ContainerDied","Data":"2de51ed0df230931e42cb701379e4adc86a0ef0d78971eb69db2179440dab486"}
Oct 30 21:56:34.499145 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:34.499009    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs" podStartSLOduration=1.49899097 podStartE2EDuration="1.49899097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:34.498410057 +0000 UTC m=+1337.170313995" watchObservedRunningTime="2026-10-30 21:56:34.49899097 +0000 UTC m=+1337.170894905"
Oct 30 21:56:35.374640 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:35.374544    5586 generic.go:356] "Generic (PLEG): container finished" podID="e54cc187-16f5-4efa-8d02-1e91b0a99d1c" containerID="dfb262670e3e32ddeaba9413a49d1c06009f48af6c3bf22568764b1db95e4109" exitCode=0
Oct 30 21:56:35.375080 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:35.374630    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs" event={"ID":"e54cc187-16f5-4efa-8d02-1e91b0a99d1c","Type":"ContainerDied","Data":"dfb262670e3e32ddeaba9413a49d1c06009f48af6c3bf22568764b1db95e4109"}
Oct 30 21:56:35.533883 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:35.533853    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"
Oct 30 21:56:35.688912 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:35.688865    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9110ee28-abf0-46dd-a2e7-deec072b5eb6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9110ee28-abf0-46dd-a2e7-deec072b5eb6-maas-api-openshift-ca\") pod \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\" (UID: \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\") "
Oct 30 21:56:35.689101 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:35.689044    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9110ee28-abf0-46dd-a2e7-deec072b5eb6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9110ee28-abf0-46dd-a2e7-deec072b5eb6-maas-api-xks-ca\") pod \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\" (UID: \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\") "
Oct 30 21:56:35.689101 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:35.689082    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9110ee28-abf0-46dd-a2e7-deec072b5eb6-kube-api-access-chsv4\" (UniqueName: \"kubernetes.io/projected/9110ee28-abf0-46dd-a2e7-deec072b5eb6-kube-api-access-chsv4\") pod \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\" (UID: \"9110ee28-abf0-46dd-a2e7-deec072b5eb6\") "
Oct 30 21:56:35.689330 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:35.689306    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9110ee28-abf0-46dd-a2e7-deec072b5eb6-maas-api-openshift-ca" pod "9110ee28-abf0-46dd-a2e7-deec072b5eb6" (UID: "9110ee28-abf0-46dd-a2e7-deec072b5eb6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:35.689896 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:35.689864    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9110ee28-abf0-46dd-a2e7-deec072b5eb6-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:35.692050 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:35.692020    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9110ee28-abf0-46dd-a2e7-deec072b5eb6-maas-api-xks-ca" pod "9110ee28-abf0-46dd-a2e7-deec072b5eb6" (UID: "9110ee28-abf0-46dd-a2e7-deec072b5eb6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:35.692473 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:35.692448    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9110ee28-abf0-46dd-a2e7-deec072b5eb6-kube-api-access-chsv4" pod "9110ee28-abf0-46dd-a2e7-deec072b5eb6" (UID: "9110ee28-abf0-46dd-a2e7-deec072b5eb6"). InnerVolumeSpecName "kube-api-access-chsv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:35.791315 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:35.791283    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9110ee28-abf0-46dd-a2e7-deec072b5eb6-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:35.791315 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:35.791312    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chsv4\" (UniqueName: \"kubernetes.io/projected/9110ee28-abf0-46dd-a2e7-deec072b5eb6-kube-api-access-chsv4\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:36.202957 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:36.202031    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:36.392819 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:36.392785    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"
Oct 30 21:56:36.392819 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:36.392799    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm" event={"ID":"9110ee28-abf0-46dd-a2e7-deec072b5eb6","Type":"ContainerDied","Data":"66d5e5470621d58ecdee59a1710777343926abc743a7cafaaaa2937ad0ce476e"}
Oct 30 21:56:36.393314 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:36.392876    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66d5e5470621d58ecdee59a1710777343926abc743a7cafaaaa2937ad0ce476e"
Oct 30 21:56:36.570968 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:36.570940    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"
Oct 30 21:56:36.709671 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:36.709626    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-kube-api-access-zj7dt\" (UniqueName: \"kubernetes.io/projected/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-kube-api-access-zj7dt\") pod \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\" (UID: \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\") "
Oct 30 21:56:36.709875 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:36.709787    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-maas-api-openshift-ca\") pod \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\" (UID: \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\") "
Oct 30 21:56:36.709953 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:36.709899    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-maas-api-xks-ca\") pod \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\" (UID: \"e54cc187-16f5-4efa-8d02-1e91b0a99d1c\") "
Oct 30 21:56:36.710276 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:36.710246    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-maas-api-openshift-ca" pod "e54cc187-16f5-4efa-8d02-1e91b0a99d1c" (UID: "e54cc187-16f5-4efa-8d02-1e91b0a99d1c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:36.710653 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:36.710627    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:36.712104 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:36.712075    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-kube-api-access-zj7dt" pod "e54cc187-16f5-4efa-8d02-1e91b0a99d1c" (UID: "e54cc187-16f5-4efa-8d02-1e91b0a99d1c"). InnerVolumeSpecName "kube-api-access-zj7dt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:36.712583 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:36.712557    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-maas-api-xks-ca" pod "e54cc187-16f5-4efa-8d02-1e91b0a99d1c" (UID: "e54cc187-16f5-4efa-8d02-1e91b0a99d1c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:36.812659 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:36.812554    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zj7dt\" (UniqueName: \"kubernetes.io/projected/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-kube-api-access-zj7dt\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:36.812659 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:36.812595    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e54cc187-16f5-4efa-8d02-1e91b0a99d1c-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:37.402315 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:37.402284    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"
Oct 30 21:56:37.402713 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:37.402292    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs" event={"ID":"e54cc187-16f5-4efa-8d02-1e91b0a99d1c","Type":"ContainerDied","Data":"490f5366148a2493120c0beb8f6fd6e23dee0b72eccb79c3f5b520995835caa2"}
Oct 30 21:56:37.402713 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:37.402417    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="490f5366148a2493120c0beb8f6fd6e23dee0b72eccb79c3f5b520995835caa2"
Oct 30 21:56:40.996562 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:40.996519    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:44.140088 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.140054    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"]
Oct 30 21:56:44.140794 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.140775    5586 cpu_manager.go:414] "containerMap: removing container" podUID="e54cc187-16f5-4efa-8d02-1e91b0a99d1c" containerName="revoke-keys"
Oct 30 21:56:44.140794 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.140795    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e54cc187-16f5-4efa-8d02-1e91b0a99d1c" containerName="revoke-keys"
Oct 30 21:56:44.140882 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.140809    5586 cpu_manager.go:414] "containerMap: removing container" podUID="9110ee28-abf0-46dd-a2e7-deec072b5eb6" containerName="revoke-keys"
Oct 30 21:56:44.140882 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.140815    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9110ee28-abf0-46dd-a2e7-deec072b5eb6" containerName="revoke-keys"
Oct 30 21:56:44.140949 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.140933    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="e54cc187-16f5-4efa-8d02-1e91b0a99d1c" containerName="revoke-keys"
Oct 30 21:56:44.140949 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.140941    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="9110ee28-abf0-46dd-a2e7-deec072b5eb6" containerName="revoke-keys"
Oct 30 21:56:44.144681 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.144663    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"
Oct 30 21:56:44.202608 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.202571    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"]
Oct 30 21:56:44.225436 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.225398    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b74bfc9b-869c-4641-98dc-091ed8bc00bb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2789fc2f-lpshv\" (UID: \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"
Oct 30 21:56:44.225617 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.225576    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbnrw\" (UniqueName: \"kubernetes.io/projected/b74bfc9b-869c-4641-98dc-091ed8bc00bb-kube-api-access-mbnrw\") pod \"maas-api-revoke-sub-2789fc2f-lpshv\" (UID: \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"
Oct 30 21:56:44.225617 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.225609    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b74bfc9b-869c-4641-98dc-091ed8bc00bb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2789fc2f-lpshv\" (UID: \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"
Oct 30 21:56:44.246037 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.246001    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:56:44.246207 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.246161    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 21:56:44.326845 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.326813    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b74bfc9b-869c-4641-98dc-091ed8bc00bb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2789fc2f-lpshv\" (UID: \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"
Oct 30 21:56:44.327078 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.326965    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbnrw\" (UniqueName: \"kubernetes.io/projected/b74bfc9b-869c-4641-98dc-091ed8bc00bb-kube-api-access-mbnrw\") pod \"maas-api-revoke-sub-2789fc2f-lpshv\" (UID: \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"
Oct 30 21:56:44.327078 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.326994    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b74bfc9b-869c-4641-98dc-091ed8bc00bb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2789fc2f-lpshv\" (UID: \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"
Oct 30 21:56:44.327694 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.327672    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b74bfc9b-869c-4641-98dc-091ed8bc00bb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2789fc2f-lpshv\" (UID: \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"
Oct 30 21:56:44.329291 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.329265    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b74bfc9b-869c-4641-98dc-091ed8bc00bb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2789fc2f-lpshv\" (UID: \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"
Oct 30 21:56:44.385025 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.384989    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbnrw\" (UniqueName: \"kubernetes.io/projected/b74bfc9b-869c-4641-98dc-091ed8bc00bb-kube-api-access-mbnrw\") pod \"maas-api-revoke-sub-2789fc2f-lpshv\" (UID: \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"
Oct 30 21:56:44.461938 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.461900    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"
Oct 30 21:56:44.846862 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:56:44.846823    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb74bfc9b_869c_4641_98dc_091ed8bc00bb.slice/crio-9053b40a8f528e1da9fd0f68c7353771fcd4748eed802acff0790d095c9014f4 WatchSource:0}: Error finding container 9053b40a8f528e1da9fd0f68c7353771fcd4748eed802acff0790d095c9014f4: Status 404 returned error can't find the container with id 9053b40a8f528e1da9fd0f68c7353771fcd4748eed802acff0790d095c9014f4
Oct 30 21:56:44.847584 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.847555    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"]
Oct 30 21:56:44.883173 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.883143    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"]
Oct 30 21:56:44.888910 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.888890    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:44.916141 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.916107    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"]
Oct 30 21:56:44.922112 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.922084    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:44.940167 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.938995    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:44.940167 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.939051    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:44.940167 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.939217    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:44.940167 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.939300    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7gr8\" (UniqueName: \"kubernetes.io/projected/d9f74a60-e9aa-4366-9c25-891da9aee43c-kube-api-access-w7gr8\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:44.940167 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.939364    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:44.940167 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.939396    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9f74a60-e9aa-4366-9c25-891da9aee43c-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:44.940167 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.939451    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:44.940167 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.939537    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:44.940167 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.939563    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:44.952762 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:44.952718    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"]
Oct 30 21:56:45.006957 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.006922    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"]
Oct 30 21:56:45.040715 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.040673    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.040902 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.040734    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.040902 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.040818    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.041034 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.040921    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.041034 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.040966    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.041034 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.040996    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.041159 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041045    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.041159 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041131    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.041264 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041185    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.041337 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041258    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.041395 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041375    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.041395 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041379    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.041504 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041448    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.041558 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041520    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.041612 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041579    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c145c160-382f-4151-bfd1-cab6a9e4dc98-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.041670 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041648    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7gr8\" (UniqueName: \"kubernetes.io/projected/d9f74a60-e9aa-4366-9c25-891da9aee43c-kube-api-access-w7gr8\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.041713 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041700    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.041768 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041747    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.041819 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041779    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9f74a60-e9aa-4366-9c25-891da9aee43c-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.041929 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041906    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2s62\" (UniqueName: \"kubernetes.io/projected/c145c160-382f-4151-bfd1-cab6a9e4dc98-kube-api-access-l2s62\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.042015 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041909    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.042015 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.041914    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.042482 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.042460    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9f74a60-e9aa-4366-9c25-891da9aee43c-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.044062 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.044040    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.044143 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.044079    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.101197 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.101110    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.101197 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.101142    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7gr8\" (UniqueName: \"kubernetes.io/projected/d9f74a60-e9aa-4366-9c25-891da9aee43c-kube-api-access-w7gr8\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.143116 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.143081    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2s62\" (UniqueName: \"kubernetes.io/projected/c145c160-382f-4151-bfd1-cab6a9e4dc98-kube-api-access-l2s62\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.143579 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.143136    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.143579 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.143187    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.143579 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.143208    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.143579 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.143328    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.143579 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.143416    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.143579 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.143460    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.143579 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.143526    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c145c160-382f-4151-bfd1-cab6a9e4dc98-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.144057 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.143602    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.144057 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.143622    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-workload-certs\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.144057 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.143755    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-credential-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.144057 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.143912    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-workload-socket\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.144301 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.144135    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-data\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.144487 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.144463    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c145c160-382f-4151-bfd1-cab6a9e4dc98-istiod-ca-cert\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.145966 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.145943    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-envoy\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.146267 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.146246    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-podinfo\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.203822 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.203791    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:45.217257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.215310    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2s62\" (UniqueName: \"kubernetes.io/projected/c145c160-382f-4151-bfd1-cab6a9e4dc98-kube-api-access-l2s62\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.217257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.216180    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-token\") pod \"e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") " pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.219310 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.219281    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"]
Oct 30 21:56:45.228258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.225285    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.235907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.235872    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"]
Oct 30 21:56:45.243145 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.242842    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.246352 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.246326    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:45.308968 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.308914    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"]
Oct 30 21:56:45.354994 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.354908    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.355612 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.355053    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.355612 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.355132    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.355612 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.355308    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.355612 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.355424    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.355612 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.355515    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.355934 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.355653    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bw7n\" (UniqueName: \"kubernetes.io/projected/fc3e1a16-31d6-4f92-b201-0672d2b02946-kube-api-access-7bw7n\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.355934 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.355907    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.356069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.356017    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.356140 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.356125    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5crgr\" (UniqueName: \"kubernetes.io/projected/a2297dba-5385-44d4-b0ae-1e470b525fc9-kube-api-access-5crgr\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.356252 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.356218    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.356329 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.356305    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2297dba-5385-44d4-b0ae-1e470b525fc9-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.356389 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.356338    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.356507 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.356488    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.356574 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.356560    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.356649 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.356588    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc3e1a16-31d6-4f92-b201-0672d2b02946-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.356753 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.356737    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.356863 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.356848    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.359966 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.359935    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"]
Oct 30 21:56:45.458121 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458081    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.458335 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458131    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.458335 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458152    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.458335 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458182    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.458335 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458251    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.458335 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458275    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.458335 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458310    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bw7n\" (UniqueName: \"kubernetes.io/projected/fc3e1a16-31d6-4f92-b201-0672d2b02946-kube-api-access-7bw7n\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.458654 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458372    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.458654 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458402    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.458654 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458430    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5crgr\" (UniqueName: \"kubernetes.io/projected/a2297dba-5385-44d4-b0ae-1e470b525fc9-kube-api-access-5crgr\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.458654 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458454    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.458654 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458475    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2297dba-5385-44d4-b0ae-1e470b525fc9-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.458654 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458498    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.458654 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458562    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.458654 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458590    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.458654 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458606    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.458654 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458616    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc3e1a16-31d6-4f92-b201-0672d2b02946-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.459157 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458687    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.459157 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458748    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.459157 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.458986    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.459363 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.459325    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.459420 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.459388    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc3e1a16-31d6-4f92-b201-0672d2b02946-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.459472 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.459416    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.460040 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.459745    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.460040 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.459969    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.460225 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.460062    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2297dba-5385-44d4-b0ae-1e470b525fc9-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.460579 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.460556    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.460727 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.460656    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.461915 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.461886    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.462189 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.462163    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.462582 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.462557    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.463323 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.463303    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.475205 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.475172    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv" event={"ID":"b74bfc9b-869c-4641-98dc-091ed8bc00bb","Type":"ContainerStarted","Data":"692cffdd6b69ac6449870e330ea934bfdf0b91a1d935d7cbbc58ddfa38427582"}
Oct 30 21:56:45.475360 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.475226    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv" event={"ID":"b74bfc9b-869c-4641-98dc-091ed8bc00bb","Type":"ContainerStarted","Data":"9053b40a8f528e1da9fd0f68c7353771fcd4748eed802acff0790d095c9014f4"}
Oct 30 21:56:45.518596 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.518528    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.519881 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.519207    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.519881 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.519807    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5crgr\" (UniqueName: \"kubernetes.io/projected/a2297dba-5385-44d4-b0ae-1e470b525fc9-kube-api-access-5crgr\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.523981 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.523726    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bw7n\" (UniqueName: \"kubernetes.io/projected/fc3e1a16-31d6-4f92-b201-0672d2b02946-kube-api-access-7bw7n\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.565635 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.565584    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:45.603642 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.603612    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:45.614561 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.614494    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv" podStartSLOduration=1.6144759720000001 podStartE2EDuration="1.614475972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:45.609299308 +0000 UTC m=+1348.281203247" watchObservedRunningTime="2026-10-30 21:56:45.614475972 +0000 UTC m=+1348.286379908"
Oct 30 21:56:45.638082 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.638035    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:45.638205 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.638130    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:45.638205 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.638170    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:45.641216 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.641191    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"]
Oct 30 21:56:45.681015 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.680036    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"]
Oct 30 21:56:45.695703 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.695657    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:45.695809 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.695760    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:45.695809 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.695804    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:45.911776 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.911735    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:45.979687 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.979642    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"]
Oct 30 21:56:45.980791 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:56:45.980754    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2297dba_5385_44d4_b0ae_1e470b525fc9.slice/crio-1a06ac9c03e79075e5c031ec4f8e2455efec17885f7063dcf3e64b603f968974 WatchSource:0}: Error finding container 1a06ac9c03e79075e5c031ec4f8e2455efec17885f7063dcf3e64b603f968974: Status 404 returned error can't find the container with id 1a06ac9c03e79075e5c031ec4f8e2455efec17885f7063dcf3e64b603f968974
Oct 30 21:56:45.983940 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.983901    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:45.984129 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.984106    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:45.984300 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:45.984202    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:46.050297 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:56:46.050257    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc3e1a16_31d6_4f92_b201_0672d2b02946.slice/crio-773e41055fd16f1f707b5d7786b42986d8b16d5a47433f20d78ad78eede64a26 WatchSource:0}: Error finding container 773e41055fd16f1f707b5d7786b42986d8b16d5a47433f20d78ad78eede64a26: Status 404 returned error can't find the container with id 773e41055fd16f1f707b5d7786b42986d8b16d5a47433f20d78ad78eede64a26
Oct 30 21:56:46.050675 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.050557    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"]
Oct 30 21:56:46.054007 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.053967    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:46.054124 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.054063    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:46.054124 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.054108    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:56:46.526316 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.525499    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" event={"ID":"d9f74a60-e9aa-4366-9c25-891da9aee43c","Type":"ContainerStarted","Data":"29b68d69c49d91bf30607ad98540ddb3439012f394ada62d82db52543455229b"}
Oct 30 21:56:46.526316 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.525567    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" event={"ID":"d9f74a60-e9aa-4366-9c25-891da9aee43c","Type":"ContainerStarted","Data":"18e5d1dfbae5252d39203fb4f7df0a0946ba1f0ec68cf2554b8f6294cc33dc6e"}
Oct 30 21:56:46.573257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.572750    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" event={"ID":"fc3e1a16-31d6-4f92-b201-0672d2b02946","Type":"ContainerStarted","Data":"fccc81d858c2daaf1cd4641d0d91067b25781a41ff3a6a42686a94f678efd869"}
Oct 30 21:56:46.573257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.572818    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" event={"ID":"fc3e1a16-31d6-4f92-b201-0672d2b02946","Type":"ContainerStarted","Data":"773e41055fd16f1f707b5d7786b42986d8b16d5a47433f20d78ad78eede64a26"}
Oct 30 21:56:46.599476 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.599435    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" event={"ID":"c145c160-382f-4151-bfd1-cab6a9e4dc98","Type":"ContainerStarted","Data":"de84582f74c1b9b6cf0e326d08d3fa53c47e6e83113534ca0ec59e30e2d31fb0"}
Oct 30 21:56:46.599719 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.599699    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" event={"ID":"c145c160-382f-4151-bfd1-cab6a9e4dc98","Type":"ContainerStarted","Data":"4a5061d2b088ae4420e12f46049e0ea97a66af624427485ed58892adc630e159"}
Oct 30 21:56:46.608855 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.608820    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" event={"ID":"a2297dba-5385-44d4-b0ae-1e470b525fc9","Type":"ContainerStarted","Data":"029c8bac358e1267ce2cc358cf70a088509f3eeb3d58e079761a769b5648d89c"}
Oct 30 21:56:46.609098 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.609080    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" event={"ID":"a2297dba-5385-44d4-b0ae-1e470b525fc9","Type":"ContainerStarted","Data":"1a06ac9c03e79075e5c031ec4f8e2455efec17885f7063dcf3e64b603f968974"}
Oct 30 21:56:46.610732 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.610709    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:56:46.612739 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.612712    5586 generic.go:356] "Generic (PLEG): container finished" podID="b74bfc9b-869c-4641-98dc-091ed8bc00bb" containerID="692cffdd6b69ac6449870e330ea934bfdf0b91a1d935d7cbbc58ddfa38427582" exitCode=0
Oct 30 21:56:46.613015 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.612868    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv" event={"ID":"b74bfc9b-869c-4641-98dc-091ed8bc00bb","Type":"ContainerDied","Data":"692cffdd6b69ac6449870e330ea934bfdf0b91a1d935d7cbbc58ddfa38427582"}
Oct 30 21:56:46.688675 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.681422    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podStartSLOduration=2.681398743 podStartE2EDuration="2.681398743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:46.664716263 +0000 UTC m=+1349.336620199" watchObservedRunningTime="2026-10-30 21:56:46.681398743 +0000 UTC m=+1349.353302682"
Oct 30 21:56:46.820048 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:46.819920    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podStartSLOduration=2.819896958 podStartE2EDuration="2.819896958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:46.783692092 +0000 UTC m=+1349.455596029" watchObservedRunningTime="2026-10-30 21:56:46.819896958 +0000 UTC m=+1349.491800895"
Oct 30 21:56:47.084195 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:47.084060    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podStartSLOduration=2.084033318 podStartE2EDuration="2.084033318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:47.054871231 +0000 UTC m=+1349.726775166" watchObservedRunningTime="2026-10-30 21:56:47.084033318 +0000 UTC m=+1349.755937259"
Oct 30 21:56:47.242261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:47.221650    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:56:47.273264 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:47.244134    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:56:47.273264 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:47.244196    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:56:47.273264 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:47.248537    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:56:47.273264 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:47.249467    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:56:47.273264 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:47.249514    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:56:47.582262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:47.581457    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:56:47.637425 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:47.637384    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:56:47.666576 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:47.666528    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:56:47.670354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:47.647867    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:56:47.695376 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:47.695321    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:56:47.891989 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:47.891876    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:56:48.141013 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:48.140858    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:56:48.148831 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:48.141655    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:48.205416 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:48.205374    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:56:48.205745 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:48.205713    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:56:48.253923 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:48.253885    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:56:48.254261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:48.254199    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:56:48.268301 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:48.268140    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:48.577259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:48.572837    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:56:48.577259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:48.572915    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:56:48.602305 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:48.583619    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:48.655274 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:48.649193    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:56:48.655274 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:48.649287    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:56:48.843082 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:56:48.842971    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:56:48.914259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:48.895626    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:49.180733 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.180674    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:49.213296 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.181886    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:56:49.274275 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.209383    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:56:49.274275 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.233583    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:56:49.274275 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.264863    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:56:49.274275 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.264927    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:56:49.274275 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.265719    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:56:49.274275 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.265771    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:56:49.627921 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.596380    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:49.627921 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.622361    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:56:49.627921 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.622423    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:56:49.627921 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.627719    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:56:49.627921 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.627784    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:56:49.637738 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.632737    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podStartSLOduration=4.632714151 podStartE2EDuration="4.632714151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:47.245504013 +0000 UTC m=+1349.917407949" watchObservedRunningTime="2026-10-30 21:56:49.632714151 +0000 UTC m=+1352.304618082"
Oct 30 21:56:49.830265 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.829752    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:49.899368 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.898955    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:56:49.899368 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.899310    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:50.043261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:49.937555    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:56:50.140439 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.140407    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:56:50.370829 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.365894    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:56:50.423390 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.423324    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:56:50.552681 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.552635    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:50.561438 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.561392    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:50.597355 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.581402    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:56:50.617262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.609310    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:56:50.642338 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.605314    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:56:50.683259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.589820    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:56:50.716258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.712319    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:56:50.734260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.589873    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:56:50.738259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.734518    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:56:50.992100 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.634360    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:56:50.992100 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.929760    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:56:50.992100 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.908949    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:56:50.992100 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.908911    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:50.992100 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:50.935391    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:56:51.427321 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:51.299706    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:56:51.427321 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:51.299768    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:56:51.427321 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:51.366000    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:56:51.427321 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:51.366059    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:56:51.427321 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:51.385615    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:51.427321 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:51.424973    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:56:51.607502 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:51.604740    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:56:51.607502 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:51.604801    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:56:51.607502 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:51.606412    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:56:51.607502 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:51.606456    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:56:51.676727 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:51.658075    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:56:51.952288 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:51.951330    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:51.952288 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:51.951476    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:56:51.952288 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:51.951749    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:56:52.098859 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.098818    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"
Oct 30 21:56:52.250354 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.224880    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:56:52.300258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.246356    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b74bfc9b-869c-4641-98dc-091ed8bc00bb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b74bfc9b-869c-4641-98dc-091ed8bc00bb-maas-api-openshift-ca\") pod \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\" (UID: \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\") "
Oct 30 21:56:52.304394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.255317    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:56:52.313508 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.313468    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:56:52.338421 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.306370    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b74bfc9b-869c-4641-98dc-091ed8bc00bb-kube-api-access-mbnrw\" (UniqueName: \"kubernetes.io/projected/b74bfc9b-869c-4641-98dc-091ed8bc00bb-kube-api-access-mbnrw\") pod \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\" (UID: \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\") "
Oct 30 21:56:52.338955 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.338917    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b74bfc9b-869c-4641-98dc-091ed8bc00bb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b74bfc9b-869c-4641-98dc-091ed8bc00bb-maas-api-xks-ca\") pod \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\" (UID: \"b74bfc9b-869c-4641-98dc-091ed8bc00bb\") "
Oct 30 21:56:52.408817 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.397640    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:52.411731 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.298475    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b74bfc9b-869c-4641-98dc-091ed8bc00bb-maas-api-openshift-ca" pod "b74bfc9b-869c-4641-98dc-091ed8bc00bb" (UID: "b74bfc9b-869c-4641-98dc-091ed8bc00bb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:52.411958 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.409455    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:52.456261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.415025    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b74bfc9b-869c-4641-98dc-091ed8bc00bb-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:52.456261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.446945    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv" event={"ID":"b74bfc9b-869c-4641-98dc-091ed8bc00bb","Type":"ContainerDied","Data":"9053b40a8f528e1da9fd0f68c7353771fcd4748eed802acff0790d095c9014f4"}
Oct 30 21:56:52.456261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.447295    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9053b40a8f528e1da9fd0f68c7353771fcd4748eed802acff0790d095c9014f4"
Oct 30 21:56:52.456261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.447218    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"
Oct 30 21:56:52.456261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.339106    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:56:52.495391 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.412098    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r" podUID="832ac4dd-f2fc-47b3-82f6-03253f91ff46" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:52.496546 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.496517    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b74bfc9b-869c-4641-98dc-091ed8bc00bb-kube-api-access-mbnrw" pod "b74bfc9b-869c-4641-98dc-091ed8bc00bb" (UID: "b74bfc9b-869c-4641-98dc-091ed8bc00bb"). InnerVolumeSpecName "kube-api-access-mbnrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:52.496782 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.496755    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:52.611260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.589365    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b74bfc9b-869c-4641-98dc-091ed8bc00bb-maas-api-xks-ca" pod "b74bfc9b-869c-4641-98dc-091ed8bc00bb" (UID: "b74bfc9b-869c-4641-98dc-091ed8bc00bb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:52.611260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.590915    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbnrw\" (UniqueName: \"kubernetes.io/projected/b74bfc9b-869c-4641-98dc-091ed8bc00bb-kube-api-access-mbnrw\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:52.611260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.590949    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b74bfc9b-869c-4641-98dc-091ed8bc00bb-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:56:52.611260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.608468    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:56:52.611260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.608533    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:56:52.626259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.614917    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:56:52.626259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.614974    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:56:52.702436 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.702378    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:52.703197 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.703045    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv" podUID="bbe524d9-e536-4990-ac0b-8ad75c9fd52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:52.718390 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.718334    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:52.971274 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.965733    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:52.971274 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:52.968272    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:53.114542 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.114404    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:53.114542 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.114470    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" podUID="146b8994-5857-43e6-a00e-40683229192f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:53.228191 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.228146    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:56:53.418506 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.418446    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:56:53.418817 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.402766    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:53.422284 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.247750    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:56:53.422527 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.422495    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:56:53.518546 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.518434    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:53.719288 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.704070    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:56:53.719288 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.704143    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:56:53.720135 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.720088    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:56:53.720364 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.720318    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:56:53.720770 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.720731    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:56:53.845927 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.845828    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:53.901470 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.895361    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:53.901470 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.895431    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n" podUID="23db1e9a-3c13-4c53-9c92-028efebcec23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:53.966411 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.966357    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:53.971145 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:53.970782    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:56:54.223828 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.223775    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:56:54.224164 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.224119    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:56:54.252273 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.250847    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:56:54.252273 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.250910    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:56:54.292516 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.292475    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:54.292819 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.292781    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" podUID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:54.334378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.334336    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:54.334672 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.334630    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" podUID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:54.423381 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.423329    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:54.519504 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.519380    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:54.655400 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.655360    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:56:54.655691 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.655659    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:56:54.664681 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.664645    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:56:54.664935 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.664907    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:56:54.705259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.684379    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:54.705259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.684449    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr" podUID="cbade6b6-381e-4034-a3c7-075ce93c8218" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:54.764330 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:54.762721    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:55.035917 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.035871    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:56:55.216799 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.054072    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:55.263576 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.263519    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:56:55.263930 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.263875    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:56:55.336020 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.335916    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:56:55.336314 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.336281    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:56:55.414449 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.414394    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:55.424427 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.424362    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" podUID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:55.437752 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.435771    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:55.530727 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.530372    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:56:55.607600 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.607506    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:56:55.607868 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.607834    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:56:55.614313 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.606279    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:56:55.614559 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.614516    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:56:55.663418 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.663373    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:55.764435 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:55.764391    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:56.216677 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.216632    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:56.241080 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.240437    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:56.253707 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.253663    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:56:56.253965 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.253930    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:56:56.305289 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.305253    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:56:56.305625 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.305572    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:56:56.349264 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.348749    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:56.454257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.448370    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:56.627747 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.540862    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:56.654648 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.552178    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:56.655008 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.566661    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:56:56.655187 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.655160    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:56:56.655332 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.622378    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:56:56.655444 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.655423    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:56:56.823488 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:56.822547    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:57.204703 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:57.204578    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:56:57.204703 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:57.204641    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:56:57.260301 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:57.240915    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:56:57.260301 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:57.246984    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:56:57.260301 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:57.247039    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:56:57.275376 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:57.272873    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:56:57.475312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:57.475080    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:57.652681 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:57.652103    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:56:57.652681 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:57.652166    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:56:57.653021 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:57.653000    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:56:57.666268 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:57.653113    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:56:57.666649 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:57.666543    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:57.854375 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:57.833349    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:58.238260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:58.207891    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:56:58.238260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:58.208061    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:56:58.260509 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:58.256131    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:56:58.260509 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:58.256201    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:56:58.260509 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:58.256344    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:56:58.290260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:58.279513    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:58.290260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:58.279774    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:56:58.492471 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:58.491292    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:56:58.696268 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:58.681417    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:56:58.696268 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:58.681493    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:56:58.730310 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:58.698336    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:58.774381 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:58.732040    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:56:58.775343 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:58.775291    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:56:58.936257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:58.934779    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:56:59.213258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.213139    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:59.235368 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.234611    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:56:59.235368 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.234688    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:56:59.270259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.259485    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:59.293264 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.261594    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:56:59.318361 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.318310    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:56:59.318627 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.281587    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:59.426614 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.418994    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:56:59.509660 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.509535    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:59.570446 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.570407    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:56:59.570738 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.570705    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:56:59.618630 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.618592    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:56:59.622259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.618847    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:56:59.712849 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.712800    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:56:59.856182 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.856068    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:59.982296 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:56:59.981354    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:57:00.252313 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.235928    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:57:00.317276 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.289815    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:57:00.317276 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.289879    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:57:00.317276 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.290919    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:57:00.317276 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.292147    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:57:00.317276 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.292192    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:57:00.321688 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.321649    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:00.538319 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.531443    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:00.571419 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.568474    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:57:00.574674 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.574637    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:57:00.575069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.575042    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:57:00.679259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.678513    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:57:00.710931 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.706375    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:57:00.710931 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.706449    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:57:00.777262 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.719466    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:00.958259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.953372    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:00.993220 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:00.991672    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:01.223325 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:01.209051    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:57:01.223325 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:01.209116    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:57:01.303261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:01.297355    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:57:01.303261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:01.297421    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:57:01.387429 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:01.335556    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:01.777252 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:01.752365    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:01.989067 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:01.989021    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:57:02.110197 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:01.999389    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:02.112625 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.001989    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:57:02.112838 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.112803    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:57:02.112947 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.058944    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:02.113069 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.090159    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:57:02.122278 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.054293    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:57:02.122278 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.121911    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:57:02.122506 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.081292    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:57:02.122636 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.122615    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:57:02.232649 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.230792    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:57:02.232649 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.230861    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:57:02.296411 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.296372    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:57:02.296983 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.296447    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:57:02.297411 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.297380    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 21:57:02.297620 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.297578    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 21:57:02.299164 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.299127    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:57:02.322827 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.305773    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:57:02.566503 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.566467    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 21:57:02.566991 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.566808    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 21:57:02.623931 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.623894    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:57:02.624197 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:02.624166    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:57:03.880258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:03.879918    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:57:03.885289 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:03.885112    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 21:57:03.937115 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:03.936577    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:57:03.939226 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:03.939114    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 21:57:03.949816 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:03.949787    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:04.210109 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:04.207304    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:57:04.210109 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:04.207356    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:04.275680 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:04.275641    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:57:04.277340 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:04.277315    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 21:57:05.003855 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:05.003802    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:57:05.005203 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:05.005181    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 21:57:08.791063 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:08.791022    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:09.007111 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:09.007066    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:13.699141 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:13.699065    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:15.225756 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:15.225720    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7060-c6a623"]
Oct 30 21:57:15.226652 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:15.226627    5586 cpu_manager.go:414] "containerMap: removing container" podUID="b74bfc9b-869c-4641-98dc-091ed8bc00bb" containerName="revoke-keys"
Oct 30 21:57:15.226764 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:15.226653    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b74bfc9b-869c-4641-98dc-091ed8bc00bb" containerName="revoke-keys"
Oct 30 21:57:15.226869 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:15.226855    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="b74bfc9b-869c-4641-98dc-091ed8bc00bb" containerName="revoke-keys"
Oct 30 21:57:15.230541 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:15.230519    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7060-c6a623"
Oct 30 21:57:15.285425 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:15.285393    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7060-c6a623"]
Oct 30 21:57:15.329791 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:15.329756    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fs5vb\""
Oct 30 21:57:15.379303 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:15.379271    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwhvx\" (UniqueName: \"kubernetes.io/projected/d49ccf5b-41d3-4f48-b66c-c22ef53d2586-kube-api-access-dwhvx\") pod \"test-curl-7060-c6a623\" (UID: \"d49ccf5b-41d3-4f48-b66c-c22ef53d2586\") " pod="opendatahub/test-curl-7060-c6a623"
Oct 30 21:57:15.481341 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:15.481208    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwhvx\" (UniqueName: \"kubernetes.io/projected/d49ccf5b-41d3-4f48-b66c-c22ef53d2586-kube-api-access-dwhvx\") pod \"test-curl-7060-c6a623\" (UID: \"d49ccf5b-41d3-4f48-b66c-c22ef53d2586\") " pod="opendatahub/test-curl-7060-c6a623"
Oct 30 21:57:15.537949 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:15.537916    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwhvx\" (UniqueName: \"kubernetes.io/projected/d49ccf5b-41d3-4f48-b66c-c22ef53d2586-kube-api-access-dwhvx\") pod \"test-curl-7060-c6a623\" (UID: \"d49ccf5b-41d3-4f48-b66c-c22ef53d2586\") " pod="opendatahub/test-curl-7060-c6a623"
Oct 30 21:57:15.542982 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:15.542950    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7060-c6a623"
Oct 30 21:57:15.926217 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:15.926188    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7060-c6a623"]
Oct 30 21:57:15.926446 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:57:15.926424    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd49ccf5b_41d3_4f48_b66c_c22ef53d2586.slice/crio-c266cd24812e5bfbb5eeeedaa8021cb7f4147cc42f187bcf2f7e309ad0012bc0 WatchSource:0}: Error finding container c266cd24812e5bfbb5eeeedaa8021cb7f4147cc42f187bcf2f7e309ad0012bc0: Status 404 returned error can't find the container with id c266cd24812e5bfbb5eeeedaa8021cb7f4147cc42f187bcf2f7e309ad0012bc0
Oct 30 21:57:15.928378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:15.928360    5586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:57:16.208973 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:16.208887    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7060-c6a623" event={"ID":"d49ccf5b-41d3-4f48-b66c-c22ef53d2586","Type":"ContainerStarted","Data":"c266cd24812e5bfbb5eeeedaa8021cb7f4147cc42f187bcf2f7e309ad0012bc0"}
Oct 30 21:57:18.371639 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:57:18.371505    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:57:18.387924 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:18.387883    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:18.591934 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:18.591901    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:19.225872 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:19.225829    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:19.260368 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:19.257939    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7060-c6a623" event={"ID":"d49ccf5b-41d3-4f48-b66c-c22ef53d2586","Type":"ContainerStarted","Data":"8e5ccb4508cc18ce09039521ed644e6cfb040a3b538e239526daae575593341c"}
Oct 30 21:57:19.372560 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:19.372504    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7060-c6a623" podStartSLOduration=1.578655997 podStartE2EDuration="4.372487388s" totalImagesPullingTime="2.793831391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:19.370922907 +0000 UTC m=+1382.042826843" watchObservedRunningTime="2026-10-30 21:57:19.372487388 +0000 UTC m=+1382.044391323"
Oct 30 21:57:19.416515 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:19.416483    5586 scope.go:122] "RemoveContainer" containerID="c383588f6978ab54ec08ffbec2a66f7fe53f5d28d749cc65f7c9135e401c95a4"
Oct 30 21:57:19.659320 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:19.659285    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:20.966022 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:20.965988    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7060-c6a623"]
Oct 30 21:57:20.986223 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:20.986195    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7060-c6a623"]
Oct 30 21:57:21.273713 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:21.273607    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7060-c6a623" podUID="d49ccf5b-41d3-4f48-b66c-c22ef53d2586" containerName="test-curl-7060-c6a623" containerID="cri-o://8e5ccb4508cc18ce09039521ed644e6cfb040a3b538e239526daae575593341c" gracePeriod=2
Oct 30 21:57:21.325520 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:57:21.325479    5586 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7060-c6a623\" is forbidden: User \"system:node:ip-10-0-130-104.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-104.us-west-1.compute.internal' and this object" podUID="d49ccf5b-41d3-4f48-b66c-c22ef53d2586" pod="opendatahub/test-curl-7060-c6a623"
Oct 30 21:57:21.691252 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:21.672892    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:21.871311 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:21.853656    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:22.130320 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:22.123499    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:23.010643 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:23.010598    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:23.159405 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:23.159189    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:23.325018 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:23.324975    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:23.353409 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:23.353362    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:23.414242 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:23.414162    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:57:23.544321 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:23.534427    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:23.777269 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:23.776744    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:24.096261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:24.074466    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:24.147261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:24.146302    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:57:24.381318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:24.361392    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:24.381318 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:24.362174    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:57:24.381983 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:24.381951    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:57:24.407810 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:24.393328    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" podUID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:24.435380 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:24.435325    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:24.474257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:24.439757    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:57:24.725722 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:24.725664    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:57:24.761139 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:24.756499    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:57:24.822293 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:24.794380    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:57:24.822293 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:24.794835    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:57:25.088509 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:25.088393    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:25.391794 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:25.391682    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:25.594383 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:25.485347    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:25.731625 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:25.731566    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:57:25.767269 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:25.731895    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:25.820645 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:25.820598    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:57:26.140407 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:26.097164    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:26.274785 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:26.274745    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/health\": context deadline exceeded"
Oct 30 21:57:26.496047 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:26.496007    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:57:26.496680 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:26.496644    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:26.567727 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:26.567675    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:57:26.601379 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:26.601321    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:26.787510 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:26.787405    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:26.808474 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:26.808433    5586 generic.go:356] "Generic (PLEG): container finished" podID="d49ccf5b-41d3-4f48-b66c-c22ef53d2586" containerID="8e5ccb4508cc18ce09039521ed644e6cfb040a3b538e239526daae575593341c" exitCode=137
Oct 30 21:57:26.822404 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:26.822357    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:57:27.203259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:27.189596    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:27.543898 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:27.543778    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:27.688823 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:27.688769    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:27.689134 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:27.689100    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:27.847320 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:27.838949    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:27.847320 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:27.839349    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:28.273268 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:28.254546    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:57:28.273268 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:28.254606    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" podUID="8310bcd1-2853-49f2-9618-0ff100c794bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:28.307264 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:28.306628    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:28.412413 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:28.412026    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/health\": context deadline exceeded"
Oct 30 21:57:29.956349 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:57:29.952703    5586 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="2.448s"
Oct 30 21:57:29.960509 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:29.960353    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:29.977251 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:29.973843    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:30.013051 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:30.013017    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:57:30.013477 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:30.013458    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:57:30.013639 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:30.013629    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:57:30.013743 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:30.013735    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:57:30.013849 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:30.013840    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:57:30.013935 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:30.013851    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7060-c6a623"
Oct 30 21:57:30.120861 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:30.120765    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:30.157211 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:30.157177    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d49ccf5b-41d3-4f48-b66c-c22ef53d2586-kube-api-access-dwhvx\" (UniqueName: \"kubernetes.io/projected/d49ccf5b-41d3-4f48-b66c-c22ef53d2586-kube-api-access-dwhvx\") pod \"d49ccf5b-41d3-4f48-b66c-c22ef53d2586\" (UID: \"d49ccf5b-41d3-4f48-b66c-c22ef53d2586\") "
Oct 30 21:57:30.160120 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:30.160087    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d49ccf5b-41d3-4f48-b66c-c22ef53d2586-kube-api-access-dwhvx" pod "d49ccf5b-41d3-4f48-b66c-c22ef53d2586" (UID: "d49ccf5b-41d3-4f48-b66c-c22ef53d2586"). InnerVolumeSpecName "kube-api-access-dwhvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:30.260383 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:30.260298    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwhvx\" (UniqueName: \"kubernetes.io/projected/d49ccf5b-41d3-4f48-b66c-c22ef53d2586-kube-api-access-dwhvx\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:57:30.414996 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:57:30.414944    5586 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7060-c6a623\" is forbidden: User \"system:node:ip-10-0-130-104.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-104.us-west-1.compute.internal' and this object" podUID="d49ccf5b-41d3-4f48-b66c-c22ef53d2586" pod="opendatahub/test-curl-7060-c6a623"
Oct 30 21:57:30.968679 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:30.968648    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7060-c6a623"
Oct 30 21:57:30.969149 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:30.968657    5586 scope.go:122] "RemoveContainer" containerID="8e5ccb4508cc18ce09039521ed644e6cfb040a3b538e239526daae575593341c"
Oct 30 21:57:31.019664 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:57:31.019624    5586 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7060-c6a623\" is forbidden: User \"system:node:ip-10-0-130-104.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-104.us-west-1.compute.internal' and this object" podUID="d49ccf5b-41d3-4f48-b66c-c22ef53d2586" pod="opendatahub/test-curl-7060-c6a623"
Oct 30 21:57:31.070399 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:57:31.070350    5586 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7060-c6a623\" is forbidden: User \"system:node:ip-10-0-130-104.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-104.us-west-1.compute.internal' and this object" podUID="d49ccf5b-41d3-4f48-b66c-c22ef53d2586" pod="opendatahub/test-curl-7060-c6a623"
Oct 30 21:57:31.450248 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:31.450198    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d49ccf5b-41d3-4f48-b66c-c22ef53d2586" path="/var/lib/kubelet/pods/d49ccf5b-41d3-4f48-b66c-c22ef53d2586/volumes"
Oct 30 21:57:32.888464 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:32.888399    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:33.139837 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:33.139737    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:36.080609 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:36.080567    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:36.277782 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:36.277739    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:36.598036 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:36.597998    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:36.760243 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:36.760204    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:36.948280 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:36.937831    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:37.422485 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.422444    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"]
Oct 30 21:57:37.423204 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.423187    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d49ccf5b-41d3-4f48-b66c-c22ef53d2586" containerName="test-curl-7060-c6a623"
Oct 30 21:57:37.423204 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.423204    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d49ccf5b-41d3-4f48-b66c-c22ef53d2586" containerName="test-curl-7060-c6a623"
Oct 30 21:57:37.423418 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.423351    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d49ccf5b-41d3-4f48-b66c-c22ef53d2586" containerName="test-curl-7060-c6a623"
Oct 30 21:57:37.426819 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.426800    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.547257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.544209    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e585c53f-openshift-default-dockercfg-svzf8\""
Oct 30 21:57:37.612322 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.592760    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:37.678461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.678356    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc730edc-8621-499c-a05a-22dc848e2727-istiod-ca-cert\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.678461 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.678430    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-istio-data\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.678750 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.678522    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrpxf\" (UniqueName: \"kubernetes.io/projected/fc730edc-8621-499c-a05a-22dc848e2727-kube-api-access-qrpxf\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.678750 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.678587    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc730edc-8621-499c-a05a-22dc848e2727-istio-token\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.678750 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.678626    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc730edc-8621-499c-a05a-22dc848e2727-istio-podinfo\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.678907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.678784    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-credential-socket\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.678957 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.678913    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-workload-certs\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.679045 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.679028    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-workload-socket\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.679788 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.679178    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-istio-envoy\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.690177 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.690138    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"]
Oct 30 21:57:37.782253 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.782186    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-workload-socket\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.785449 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.785411    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-istio-envoy\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.785655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.785537    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc730edc-8621-499c-a05a-22dc848e2727-istiod-ca-cert\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.785655 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.785578    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-istio-data\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.785799 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.785657    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrpxf\" (UniqueName: \"kubernetes.io/projected/fc730edc-8621-499c-a05a-22dc848e2727-kube-api-access-qrpxf\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.785799 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.785701    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc730edc-8621-499c-a05a-22dc848e2727-istio-token\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.785799 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.785723    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc730edc-8621-499c-a05a-22dc848e2727-istio-podinfo\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.785799 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.785799    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-credential-socket\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.786211 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.785858    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-workload-certs\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.786341 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.786300    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-workload-certs\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.786341 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.782804    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-workload-socket\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.789631 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.787762    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc730edc-8621-499c-a05a-22dc848e2727-istiod-ca-cert\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.789631 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.788028    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-istio-data\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.789631 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.788184    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-credential-socket\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.789631 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.789163    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-istio-envoy\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.793499 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.793394    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc730edc-8621-499c-a05a-22dc848e2727-istio-podinfo\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.821776 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.821737    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:37.850298 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.848842    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrpxf\" (UniqueName: \"kubernetes.io/projected/fc730edc-8621-499c-a05a-22dc848e2727-kube-api-access-qrpxf\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:37.850298 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:37.849428    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc730edc-8621-499c-a05a-22dc848e2727-istio-token\") pod \"e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") " pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:38.042006 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:38.041902    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:38.492737 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:38.492704    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"]
Oct 30 21:57:38.493070 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:57:38.492735    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc730edc_8621_499c_a05a_22dc848e2727.slice/crio-3929289ec4bccbfc41040b94908e80ed2c473fea8d9c9fff3baa538e46d6a6c9 WatchSource:0}: Error finding container 3929289ec4bccbfc41040b94908e80ed2c473fea8d9c9fff3baa538e46d6a6c9: Status 404 returned error can't find the container with id 3929289ec4bccbfc41040b94908e80ed2c473fea8d9c9fff3baa538e46d6a6c9
Oct 30 21:57:38.495507 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:38.495478    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:57:38.495591 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:38.495539    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:57:38.495591 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:38.495567    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:57:38.835132 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:38.835045    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:39.085247 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:39.085132    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" event={"ID":"fc730edc-8621-499c-a05a-22dc848e2727","Type":"ContainerStarted","Data":"28ac2fcb43e22664271a4cf7e1bd5b9095f54aaf986de3e92293c61bc067b73e"}
Oct 30 21:57:39.085247 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:39.085196    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" event={"ID":"fc730edc-8621-499c-a05a-22dc848e2727","Type":"ContainerStarted","Data":"3929289ec4bccbfc41040b94908e80ed2c473fea8d9c9fff3baa538e46d6a6c9"}
Oct 30 21:57:39.256580 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:39.256520    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" podStartSLOduration=2.256504514 podStartE2EDuration="2.256504514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:39.252514997 +0000 UTC m=+1401.924418929" watchObservedRunningTime="2026-10-30 21:57:39.256504514 +0000 UTC m=+1401.928408475"
Oct 30 21:57:40.043061 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:40.043031    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:40.048815 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:40.048790    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:40.093171 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:40.093142    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:40.094052 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:40.094031    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:57:41.329765 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:41.329726    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:41.619952 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:41.619845    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:44.102361 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:44.102318    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:44.575753 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:44.575714    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:44.756774 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:44.755540    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:45.422478 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.422436    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"]
Oct 30 21:57:45.431706 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.431676    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.436448 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.436417    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"]
Oct 30 21:57:45.484250 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.484203    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4tfh8\""
Oct 30 21:57:45.533645 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.533610    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 21:57:45.585748 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.585711    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.585958 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.585922    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.586021 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.585952    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.586021 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.585990    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.586116 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.586033    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.586787 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.586159    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pr4f\" (UniqueName: \"kubernetes.io/projected/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-kube-api-access-4pr4f\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.586787 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.586272    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.587774 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.587750    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 21:57:45.587907 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.587864    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 21:57:45.688544 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.688443    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.688544 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.688492    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.688544 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.688528    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.688825 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.688553    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.688825 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.688640    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pr4f\" (UniqueName: \"kubernetes.io/projected/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-kube-api-access-4pr4f\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.688825 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.688760    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.688975 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.688935    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.689455 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.689426    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.691145 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.691113    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.691339 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.691323    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.691393 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.691369    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.691439 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.691402    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.751242 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.751193    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pr4f\" (UniqueName: \"kubernetes.io/projected/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-kube-api-access-4pr4f\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.751383 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.751270    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-4z8hd\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:45.756486 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:45.756459    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:46.220145 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:57:46.219306    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b3d579e_7b2e_46bb_87a5_0bc19e8d00fc.slice/crio-8e383fc67d673554a2df1b3178fd3694a61614d30509d2ab4326fdb5e57b6efe WatchSource:0}: Error finding container 8e383fc67d673554a2df1b3178fd3694a61614d30509d2ab4326fdb5e57b6efe: Status 404 returned error can't find the container with id 8e383fc67d673554a2df1b3178fd3694a61614d30509d2ab4326fdb5e57b6efe
Oct 30 21:57:46.236369 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:46.236336    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"]
Oct 30 21:57:47.217256 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:47.213280    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd" event={"ID":"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc","Type":"ContainerStarted","Data":"8e383fc67d673554a2df1b3178fd3694a61614d30509d2ab4326fdb5e57b6efe"}
Oct 30 21:57:48.435274 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:48.435129    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:48.595042 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:57:48.594956    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:57:48.608554 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:48.607345    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:48.682309 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:48.677368    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:48.805839 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:48.805744    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:48.847614 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:48.847564    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:49.113333 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:49.112279    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:57:49.584260 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:49.583941    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:49.626395 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:49.610374    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:49.839312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:49.833337    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:49.839312 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:49.833602    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:49.857958 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:49.853872    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:49.970983 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:49.964013    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:57:49.970983 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:49.964085    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:50.121365 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:50.116329    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:57:50.998332 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:57:50.998289    5586 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.547s"
Oct 30 21:57:50.999498 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:50.999460    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:57:51.077860 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:51.059647    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:57:51.077860 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:51.059735    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:57:51.077860 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:51.060004    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:57:51.077860 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:51.065618    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:57:51.077860 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:51.065737    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:51.114255 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:51.088649    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 21:57:51.345314 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:51.332625    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:51.347272 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:51.346796    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:57:51.398272 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:51.395310    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:57:51.449266 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:51.443311    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 21:57:51.449266 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:51.445254    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:57:51.653319 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:51.606165    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:51.726332 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:51.726285    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:57:51.726711 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:51.726672    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:52.129220 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:52.129175    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-tksjb" podUID="4c8745d6-d399-4ebf-ac9a-856cfae9f873" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:52.240456 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:52.240118    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:57:52.240456 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:52.240243    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:57:52.240456 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:52.240288    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:57:52.240456 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:52.240326    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:57:52.240456 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:52.240360    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:57:52.405668 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:52.405585    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:55.105651 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:55.105603    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:57:55.106057 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:55.105693    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:57:55.245324 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:55.245288    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd" event={"ID":"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc","Type":"ContainerStarted","Data":"eac86fb5028cf63f0da6909e325dbdb5cddfbf48d532a66d299a08472aebca62"}
Oct 30 21:57:55.245603 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:55.245515    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:57:55.376019 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:55.375896    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd" podStartSLOduration=1.500030356 podStartE2EDuration="10.375875745s" totalImagesPullingTime="8.875845389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:55.372265752 +0000 UTC m=+1418.044169691" watchObservedRunningTime="2026-10-30 21:57:55.375875745 +0000 UTC m=+1418.047779681"
Oct 30 21:57:55.971878 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:55.971823    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:56.257730 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:56.257635    5586 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-4z8hd container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:57:56.257730 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:56.257690    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd" podUID="1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:57:56.519964 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:56.519861    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:59.256045 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:57:59.256012    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 21:58:00.830180 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:00.830140    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:05.000305 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:05.000267    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:05.187737 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:05.187684    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:05.367067 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:05.366972    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:05.627542 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:05.627451    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:08.732849 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:08.732804    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:10.442374 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:10.442338    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:10.618984 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:10.618951    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:10.802495 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:10.802409    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:11.173126 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:11.173086    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:11.427177 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:11.427088    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:11.665331 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:11.665294    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:15.820166 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:15.820129    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:18.174211 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:18.174177    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:18.368693 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:58:18.368658    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:58:19.848134 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:19.848034    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:20.093492 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:20.093453    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:20.497600 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:20.497557    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:20.995679 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:20.995648    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:21.210914 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:21.210880    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:21.383988 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:21.383893    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:23.787699 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:23.787658    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:25.027157 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:25.027119    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:26.204404 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:26.204363    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:27.244860 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:27.244823    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:27.397930 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:27.397892    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:31.192364 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:31.192267    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:32.020272 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.020221    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"]
Oct 30 21:58:32.030570 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.030539    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.131308 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.131268    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"]
Oct 30 21:58:32.158295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.154129    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.158295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.154270    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.158295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.154369    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.158295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.154467    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.158295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.154575    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.158295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.154634    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.158295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.154747    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h2tp\" (UniqueName: \"kubernetes.io/projected/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-kube-api-access-9h2tp\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.158295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.154804    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.158295 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.154872    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.256011 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.255976    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.256514 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.256039    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.256514 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.256107    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h2tp\" (UniqueName: \"kubernetes.io/projected/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-kube-api-access-9h2tp\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.256514 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.256150    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.256514 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.256215    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.256514 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.256303    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.256514 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.256345    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.256853 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.256535    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.256853 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.256578    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.256853 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.256680    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.256853 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.256770    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.256853 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.256680    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.257065 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.256907    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.257138 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.257120    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.258771 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.258748    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.259110 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.259088    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.315681 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.315592    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h2tp\" (UniqueName: \"kubernetes.io/projected/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-kube-api-access-9h2tp\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.316609 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.316584    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-nzzv2\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.345877 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.345841    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:32.856199 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.856160    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"]
Oct 30 21:58:32.887820 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:58:32.887772    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc20087fb_273e_4ba6_bc41_3a31cfeb30ec.slice/crio-673fecacbed66efc276a13979026deb44b4b5ee11c8a8447c46233c036debe79 WatchSource:0}: Error finding container 673fecacbed66efc276a13979026deb44b4b5ee11c8a8447c46233c036debe79: Status 404 returned error can't find the container with id 673fecacbed66efc276a13979026deb44b4b5ee11c8a8447c46233c036debe79
Oct 30 21:58:32.891168 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.891125    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:58:32.891308 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.891256    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:58:32.891372 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:32.891306    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:58:33.582019 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:33.581982    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" event={"ID":"c20087fb-273e-4ba6-bc41-3a31cfeb30ec","Type":"ContainerStarted","Data":"cb8d9ac20498f562a95b04a572c27bff38ee85cd9b5abb150b9f215a613449d7"}
Oct 30 21:58:33.582444 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:33.582033    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" event={"ID":"c20087fb-273e-4ba6-bc41-3a31cfeb30ec","Type":"ContainerStarted","Data":"673fecacbed66efc276a13979026deb44b4b5ee11c8a8447c46233c036debe79"}
Oct 30 21:58:33.704380 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:33.704305    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" podStartSLOduration=2.704287688 podStartE2EDuration="2.704287688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:58:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:58:33.701899869 +0000 UTC m=+1456.373803805" watchObservedRunningTime="2026-10-30 21:58:33.704287688 +0000 UTC m=+1456.376191623"
Oct 30 21:58:34.346491 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:34.346445    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:34.348549 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:34.348508    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:58:34.348701 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:34.348563    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" podUID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 21:58:35.351391 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:35.351338    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:58:35.352024 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:35.351989    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" podUID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 21:58:36.127269 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:36.112656    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:36.409268 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:36.406598    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:58:36.409268 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:36.406694    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" podUID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 21:58:36.653122 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:36.653082    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:36.890509 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:36.890411    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:37.246253 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:37.245377    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:37.290577 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:37.290539    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:37.351940 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:37.351492    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:58:37.351940 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:37.351552    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" podUID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 21:58:37.500321 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:37.489382    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:38.061268 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:38.061037    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:38.119258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:38.118400    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:58:38.119258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:38.118466    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" podUID="146b8994-5857-43e6-a00e-40683229192f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:38.155261 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:38.146345    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:38.204740 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:38.204702    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:38.258431 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:38.258381    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:38.272378 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:38.272332    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:38.279371 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:38.279334    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:58:38.350893 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:38.350796    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:58:38.351157 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:38.351128    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" podUID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 21:58:38.422955 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:38.422908    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:38.575259 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:38.549870    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:38.591889 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:38.591850    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:58:39.154716 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.154658    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:39.155613 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.155587    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:58:39.220265 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.206387    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:39.220265 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.207035    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:58:39.289042 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.288998    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:39.352306 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.348643    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:58:39.352306 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.348710    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" podUID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 21:58:39.427599 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.427499    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:39.433047 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.428196    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:58:39.578446 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.578390    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:39.621419 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.621368    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:58:39.621760 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.621687    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr" podUID="1a027617-89f1-4e5e-84f0-4bc510c9b357" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:58:39.835062 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.834961    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/health\": context deadline exceeded"
Oct 30 21:58:39.982590 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.982314    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 21:58:39.982590 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.982367    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 21:58:39.991904 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:39.991801    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 21:58:40.005623 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:40.005579    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 21:58:40.017675 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:40.017604    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 21:58:40.349265 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:40.348441    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:58:40.349265 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:40.348510    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" podUID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 21:58:41.347063 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:41.347023    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:58:41.347516 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:41.347086    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" podUID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 21:58:41.573224 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:41.573178    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:42.346444 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:42.346401    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-nzzv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 30 21:58:42.346634 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:42.346459    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" podUID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 30 21:58:43.896672 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:43.896633    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:43.897172 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:43.897075    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:43.898660 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:43.898636    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 21:58:46.496342 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:46.496305    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:48.362397 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:58:48.362359    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:58:50.437335 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:50.437295    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:50.649830 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:50.649791    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:51.385388 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:51.385348    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:56.401927 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:58:56.401888    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:01.453837 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:01.453796    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:02.376072 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.376040    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"]
Oct 30 21:59:02.380995 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.380977    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.468514 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.468463    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.469013 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.468527    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.469013 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.468640    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.469013 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.468708    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.469189 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.469088    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcv4v\" (UniqueName: \"kubernetes.io/projected/fe4d37c0-63a4-4c62-a02b-97df821badbb-kube-api-access-zcv4v\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.469189 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.469129    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4d37c0-63a4-4c62-a02b-97df821badbb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.469327 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.469292    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.469385 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.469360    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.469549 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.469513    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.487452 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.487412    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"]
Oct 30 21:59:02.571472 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.571425    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.571677 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.571541    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.571677 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.571590    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.571677 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.571634    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.571677 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.571656    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcv4v\" (UniqueName: \"kubernetes.io/projected/fe4d37c0-63a4-4c62-a02b-97df821badbb-kube-api-access-zcv4v\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.571912 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.571684    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4d37c0-63a4-4c62-a02b-97df821badbb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.571912 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.571746    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.571912 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.571804    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.571912 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.571886    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.572171 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.571931    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.572171 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.572044    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.572171 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.572102    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.572394 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.572190    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.572504 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.572484    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4d37c0-63a4-4c62-a02b-97df821badbb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.573884 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.573856    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.574419 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.574401    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.638171 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.638098    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcv4v\" (UniqueName: \"kubernetes.io/projected/fe4d37c0-63a4-4c62-a02b-97df821badbb-kube-api-access-zcv4v\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.640641 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.640611    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-fwnc4\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:02.699434 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:02.699397    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:03.157378 ip-10-0-130-104 kubenswrapper[5586]: W1030 21:59:03.157345    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4d37c0_63a4_4c62_a02b_97df821badbb.slice/crio-1bdbc8db7f831aa41c2c814f7951f65c43a8355fc2081dd187672b9b3a04060b WatchSource:0}: Error finding container 1bdbc8db7f831aa41c2c814f7951f65c43a8355fc2081dd187672b9b3a04060b: Status 404 returned error can't find the container with id 1bdbc8db7f831aa41c2c814f7951f65c43a8355fc2081dd187672b9b3a04060b
Oct 30 21:59:03.157870 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:03.157843    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"]
Oct 30 21:59:03.160415 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:03.160383    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:03.160508 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:03.160451    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:03.160508 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:03.160478    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:03.291460 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:03.291418    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:03.525217 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:03.525167    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4" event={"ID":"fe4d37c0-63a4-4c62-a02b-97df821badbb","Type":"ContainerStarted","Data":"e473ada97c26beef41cc089ea5f07b72598064f0dadb79f64140be281e895298"}
Oct 30 21:59:03.525711 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:03.525261    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4" event={"ID":"fe4d37c0-63a4-4c62-a02b-97df821badbb","Type":"ContainerStarted","Data":"1bdbc8db7f831aa41c2c814f7951f65c43a8355fc2081dd187672b9b3a04060b"}
Oct 30 21:59:03.654501 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:03.654439    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4" podStartSLOduration=1.654424681 podStartE2EDuration="1.654424681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:03.65231468 +0000 UTC m=+1486.324218633" watchObservedRunningTime="2026-10-30 21:59:03.654424681 +0000 UTC m=+1486.326328615"
Oct 30 21:59:03.699897 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:03.699861    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:04.701392 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:04.701353    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 21:59:04.701808 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:04.701424    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4" podUID="fe4d37c0-63a4-4c62-a02b-97df821badbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 21:59:05.264325 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:05.264288    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:05.700563 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:05.700525    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 21:59:05.700759 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:05.700586    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4" podUID="fe4d37c0-63a4-4c62-a02b-97df821badbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 21:59:06.699792 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:06.699751    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 21:59:06.700203 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:06.699810    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4" podUID="fe4d37c0-63a4-4c62-a02b-97df821badbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 21:59:07.107995 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:07.107903    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:07.700484 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:07.700451    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 21:59:07.700910 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:07.700519    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4" podUID="fe4d37c0-63a4-4c62-a02b-97df821badbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 21:59:08.700633 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:08.700594    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 21:59:08.701085 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:08.700664    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4" podUID="fe4d37c0-63a4-4c62-a02b-97df821badbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 21:59:09.700585 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:09.700550    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 21:59:09.700809 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:09.700619    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4" podUID="fe4d37c0-63a4-4c62-a02b-97df821badbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 21:59:10.700443 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:10.700407    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-fwnc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Oct 30 21:59:10.700637 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:10.700472    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4" podUID="fe4d37c0-63a4-4c62-a02b-97df821badbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Oct 30 21:59:12.116547 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:12.116508    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:12.117113 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:12.117089    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:12.117956 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:12.117933    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 21:59:12.683258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:12.683211    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:17.722790 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:17.722761    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 21:59:17.724421 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:17.724391    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 21:59:17.750362 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:17.750326    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 21:59:17.750362 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:17.750326    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 21:59:18.398315 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:59:18.398277    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:59:18.667396 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:18.667358    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:24.085893 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:24.085851    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:24.437653 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:24.437600    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:26.135401 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:26.135369    5586 patch_prober.go:29] interesting pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:59:26.135884 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:26.135430    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" podUID="fc730edc-8621-499c-a05a-22dc848e2727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:59:28.725253 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:28.725176    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:29.836958 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:29.836864    5586 generic.go:356] "Generic (PLEG): container finished" podID="fc730edc-8621-499c-a05a-22dc848e2727" containerID="28ac2fcb43e22664271a4cf7e1bd5b9095f54aaf986de3e92293c61bc067b73e" exitCode=0
Oct 30 21:59:29.836958 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:29.836930    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" event={"ID":"fc730edc-8621-499c-a05a-22dc848e2727","Type":"ContainerDied","Data":"28ac2fcb43e22664271a4cf7e1bd5b9095f54aaf986de3e92293c61bc067b73e"}
Oct 30 21:59:29.837488 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:29.837472    5586 scope.go:122] "RemoveContainer" containerID="28ac2fcb43e22664271a4cf7e1bd5b9095f54aaf986de3e92293c61bc067b73e"
Oct 30 21:59:29.839330 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:29.839299    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:29.839451 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:29.839369    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:29.839451 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:29.839399    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 21:59:29.974436 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:29.974398    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:30.611172 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:30.611117    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:30.855286 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:30.855251    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" event={"ID":"fc730edc-8621-499c-a05a-22dc848e2727","Type":"ContainerStarted","Data":"186ccdae1e3b06c02e1d200aa92dd30a79aaf1a759aff1d3842b4e86fb484832"}
Oct 30 21:59:31.047862 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:31.047347    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:59:31.047862 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:31.047505    5586 patch_prober.go:29] interesting pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:31.047862 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:31.047556    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" podUID="fc730edc-8621-499c-a05a-22dc848e2727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 21:59:32.047200 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:32.046759    5586 patch_prober.go:29] interesting pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:32.047200 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:32.046831    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" podUID="fc730edc-8621-499c-a05a-22dc848e2727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 21:59:33.045258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:33.044700    5586 patch_prober.go:29] interesting pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:33.045258 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:33.044784    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" podUID="fc730edc-8621-499c-a05a-22dc848e2727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 21:59:34.046786 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:34.046750    5586 patch_prober.go:29] interesting pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:34.047332 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:34.047304    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" podUID="fc730edc-8621-499c-a05a-22dc848e2727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 21:59:35.043268 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:35.042937    5586 patch_prober.go:29] interesting pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:35.043268 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:35.043004    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" podUID="fc730edc-8621-499c-a05a-22dc848e2727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 21:59:35.384434 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:35.383872    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:36.046408 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:36.042707    5586 patch_prober.go:29] interesting pod/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:59:36.046408 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:36.042778    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" podUID="fc730edc-8621-499c-a05a-22dc848e2727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 21:59:37.009870 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:37.009821    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:37.047205 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:37.047171    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:59:37.047606 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:37.047589    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:59:37.048515 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:37.048493    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 21:59:37.168750 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:37.168701    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:40.774257 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:40.774031    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:46.110547 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:46.110511    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:48.363589 ip-10-0-130-104 kubenswrapper[5586]: E1030 21:59:48.363554    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 21:59:50.991465 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:50.991418    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:52.190872 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:52.190829    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:52.408865 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:52.408827    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:55.893842 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:55.893799    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:59.748580 ip-10-0-130-104 kubenswrapper[5586]: I1030 21:59:59.748492    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:00.242021 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.241975    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"]
Oct 30 22:00:00.245748 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.245725    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"
Oct 30 22:00:00.246697 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.246671    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"]
Oct 30 22:00:00.250734 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.250709    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"
Oct 30 22:00:00.301858 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.301819    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11ca645b-a522-40ff-bd25-c1ee49f02e58-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n\" (UID: \"11ca645b-a522-40ff-bd25-c1ee49f02e58\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"
Oct 30 22:00:00.302064 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.301955    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsvmt\" (UniqueName: \"kubernetes.io/projected/f2822b90-308a-4149-a838-3e23dccbcb62-kube-api-access-fsvmt\") pod \"maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2\" (UID: \"f2822b90-308a-4149-a838-3e23dccbcb62\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"
Oct 30 22:00:00.302154 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.302119    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kzfs\" (UniqueName: \"kubernetes.io/projected/11ca645b-a522-40ff-bd25-c1ee49f02e58-kube-api-access-8kzfs\") pod \"maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n\" (UID: \"11ca645b-a522-40ff-bd25-c1ee49f02e58\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"
Oct 30 22:00:00.302320 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.302303    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2822b90-308a-4149-a838-3e23dccbcb62-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2\" (UID: \"f2822b90-308a-4149-a838-3e23dccbcb62\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"
Oct 30 22:00:00.327315 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.327273    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:00:00.333773 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.333737    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"]
Oct 30 22:00:00.338593 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.338568    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"]
Oct 30 22:00:00.338772 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.338724    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"
Oct 30 22:00:00.351514 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.351084    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"
Oct 30 22:00:00.384807 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.384773    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"]
Oct 30 22:00:00.392418 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.392390    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"]
Oct 30 22:00:00.396055 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.396034    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"
Oct 30 22:00:00.400012 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.399983    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"]
Oct 30 22:00:00.403923 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.403893    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4sb5\" (UniqueName: \"kubernetes.io/projected/78ef55a3-8cd6-4b4d-9231-0a9745440fc6-kube-api-access-c4sb5\") pod \"maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn\" (UID: \"78ef55a3-8cd6-4b4d-9231-0a9745440fc6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"
Oct 30 22:00:00.404620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.403963    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2822b90-308a-4149-a838-3e23dccbcb62-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2\" (UID: \"f2822b90-308a-4149-a838-3e23dccbcb62\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"
Oct 30 22:00:00.404620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.404006    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b31903-7a3a-4f41-b955-764d32eafb50-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889960-gq2xq\" (UID: \"26b31903-7a3a-4f41-b955-764d32eafb50\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"
Oct 30 22:00:00.404620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.404140    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11ca645b-a522-40ff-bd25-c1ee49f02e58-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n\" (UID: \"11ca645b-a522-40ff-bd25-c1ee49f02e58\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"
Oct 30 22:00:00.404620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.404227    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsvmt\" (UniqueName: \"kubernetes.io/projected/f2822b90-308a-4149-a838-3e23dccbcb62-kube-api-access-fsvmt\") pod \"maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2\" (UID: \"f2822b90-308a-4149-a838-3e23dccbcb62\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"
Oct 30 22:00:00.404620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.404378    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78ef55a3-8cd6-4b4d-9231-0a9745440fc6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn\" (UID: \"78ef55a3-8cd6-4b4d-9231-0a9745440fc6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"
Oct 30 22:00:00.404620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.404438    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fg4g\" (UniqueName: \"kubernetes.io/projected/26b31903-7a3a-4f41-b955-764d32eafb50-kube-api-access-8fg4g\") pod \"maas-api-key-cleanup-29889960-gq2xq\" (UID: \"26b31903-7a3a-4f41-b955-764d32eafb50\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"
Oct 30 22:00:00.404620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.404521    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kzfs\" (UniqueName: \"kubernetes.io/projected/11ca645b-a522-40ff-bd25-c1ee49f02e58-kube-api-access-8kzfs\") pod \"maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n\" (UID: \"11ca645b-a522-40ff-bd25-c1ee49f02e58\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"
Oct 30 22:00:00.405802 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.405777    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11ca645b-a522-40ff-bd25-c1ee49f02e58-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n\" (UID: \"11ca645b-a522-40ff-bd25-c1ee49f02e58\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"
Oct 30 22:00:00.405915 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.405898    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2822b90-308a-4149-a838-3e23dccbcb62-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2\" (UID: \"f2822b90-308a-4149-a838-3e23dccbcb62\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"
Oct 30 22:00:00.425807 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.425772    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"]
Oct 30 22:00:00.434362 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.433935    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"]
Oct 30 22:00:00.441158 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.441126    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"]
Oct 30 22:00:00.471830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.471798    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kzfs\" (UniqueName: \"kubernetes.io/projected/11ca645b-a522-40ff-bd25-c1ee49f02e58-kube-api-access-8kzfs\") pod \"maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n\" (UID: \"11ca645b-a522-40ff-bd25-c1ee49f02e58\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"
Oct 30 22:00:00.494109 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.494032    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsvmt\" (UniqueName: \"kubernetes.io/projected/f2822b90-308a-4149-a838-3e23dccbcb62-kube-api-access-fsvmt\") pod \"maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2\" (UID: \"f2822b90-308a-4149-a838-3e23dccbcb62\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"
Oct 30 22:00:00.506577 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.506549    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78ef55a3-8cd6-4b4d-9231-0a9745440fc6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn\" (UID: \"78ef55a3-8cd6-4b4d-9231-0a9745440fc6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"
Oct 30 22:00:00.506721 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.506586    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct22r\" (UniqueName: \"kubernetes.io/projected/0561cf7c-914f-4660-8b42-ecb3c42bec98-kube-api-access-ct22r\") pod \"maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6\" (UID: \"0561cf7c-914f-4660-8b42-ecb3c42bec98\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"
Oct 30 22:00:00.506721 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.506626    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fg4g\" (UniqueName: \"kubernetes.io/projected/26b31903-7a3a-4f41-b955-764d32eafb50-kube-api-access-8fg4g\") pod \"maas-api-key-cleanup-29889960-gq2xq\" (UID: \"26b31903-7a3a-4f41-b955-764d32eafb50\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"
Oct 30 22:00:00.506844 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.506822    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4sb5\" (UniqueName: \"kubernetes.io/projected/78ef55a3-8cd6-4b4d-9231-0a9745440fc6-kube-api-access-c4sb5\") pod \"maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn\" (UID: \"78ef55a3-8cd6-4b4d-9231-0a9745440fc6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"
Oct 30 22:00:00.506913 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.506883    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b31903-7a3a-4f41-b955-764d32eafb50-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889960-gq2xq\" (UID: \"26b31903-7a3a-4f41-b955-764d32eafb50\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"
Oct 30 22:00:00.507070 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.507050    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0561cf7c-914f-4660-8b42-ecb3c42bec98-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6\" (UID: \"0561cf7c-914f-4660-8b42-ecb3c42bec98\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"
Oct 30 22:00:00.507442 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.507419    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78ef55a3-8cd6-4b4d-9231-0a9745440fc6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn\" (UID: \"78ef55a3-8cd6-4b4d-9231-0a9745440fc6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"
Oct 30 22:00:00.507556 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.507539    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b31903-7a3a-4f41-b955-764d32eafb50-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889960-gq2xq\" (UID: \"26b31903-7a3a-4f41-b955-764d32eafb50\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"
Oct 30 22:00:00.561746 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.561709    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"
Oct 30 22:00:00.569040 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.569008    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"
Oct 30 22:00:00.569497 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.569476    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4sb5\" (UniqueName: \"kubernetes.io/projected/78ef55a3-8cd6-4b4d-9231-0a9745440fc6-kube-api-access-c4sb5\") pod \"maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn\" (UID: \"78ef55a3-8cd6-4b4d-9231-0a9745440fc6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"
Oct 30 22:00:00.573635 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.573608    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fg4g\" (UniqueName: \"kubernetes.io/projected/26b31903-7a3a-4f41-b955-764d32eafb50-kube-api-access-8fg4g\") pod \"maas-api-key-cleanup-29889960-gq2xq\" (UID: \"26b31903-7a3a-4f41-b955-764d32eafb50\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"
Oct 30 22:00:00.609953 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.609909    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0561cf7c-914f-4660-8b42-ecb3c42bec98-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6\" (UID: \"0561cf7c-914f-4660-8b42-ecb3c42bec98\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"
Oct 30 22:00:00.610564 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.610541    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0561cf7c-914f-4660-8b42-ecb3c42bec98-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6\" (UID: \"0561cf7c-914f-4660-8b42-ecb3c42bec98\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"
Oct 30 22:00:00.610739 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.610721    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct22r\" (UniqueName: \"kubernetes.io/projected/0561cf7c-914f-4660-8b42-ecb3c42bec98-kube-api-access-ct22r\") pod \"maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6\" (UID: \"0561cf7c-914f-4660-8b42-ecb3c42bec98\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"
Oct 30 22:00:00.659576 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.659351    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"
Oct 30 22:00:00.671121 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.671096    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"
Oct 30 22:00:00.681689 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.681630    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct22r\" (UniqueName: \"kubernetes.io/projected/0561cf7c-914f-4660-8b42-ecb3c42bec98-kube-api-access-ct22r\") pod \"maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6\" (UID: \"0561cf7c-914f-4660-8b42-ecb3c42bec98\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"
Oct 30 22:00:00.711772 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.711732    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"
Oct 30 22:00:00.965983 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:00:00.965948    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2822b90_308a_4149_a838_3e23dccbcb62.slice/crio-f0425b40a39261a6a3758a82adffa5c3ef103fb1a499c7ed8eff95dd7c798761 WatchSource:0}: Error finding container f0425b40a39261a6a3758a82adffa5c3ef103fb1a499c7ed8eff95dd7c798761: Status 404 returned error can't find the container with id f0425b40a39261a6a3758a82adffa5c3ef103fb1a499c7ed8eff95dd7c798761
Oct 30 22:00:00.973375 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:00.973345    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"]
Oct 30 22:00:01.008516 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:01.008468    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:01.045992 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:01.045936    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"]
Oct 30 22:00:01.126724 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:01.126056    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"]
Oct 30 22:00:01.128377 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:00:01.128339    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26b31903_7a3a_4f41_b955_764d32eafb50.slice/crio-cafc4946ce9f0cbfbf702f69ee11608e629e37ea7cea18a8daee08b247e76249 WatchSource:0}: Error finding container cafc4946ce9f0cbfbf702f69ee11608e629e37ea7cea18a8daee08b247e76249: Status 404 returned error can't find the container with id cafc4946ce9f0cbfbf702f69ee11608e629e37ea7cea18a8daee08b247e76249
Oct 30 22:00:01.129426 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:00:01.129403    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78ef55a3_8cd6_4b4d_9231_0a9745440fc6.slice/crio-939ee9dbf09c5ecd3c92e5e0d812061386b958ff655896a90470f3b8a0ea1ecc WatchSource:0}: Error finding container 939ee9dbf09c5ecd3c92e5e0d812061386b958ff655896a90470f3b8a0ea1ecc: Status 404 returned error can't find the container with id 939ee9dbf09c5ecd3c92e5e0d812061386b958ff655896a90470f3b8a0ea1ecc
Oct 30 22:00:01.136571 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:01.136126    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n" event={"ID":"11ca645b-a522-40ff-bd25-c1ee49f02e58","Type":"ContainerStarted","Data":"69ef1ce83df2f48b64390724b3c59fa7b7365dd6a0b9491ee439674c732a9ec8"}
Oct 30 22:00:01.138152 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:01.138128    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2" event={"ID":"f2822b90-308a-4149-a838-3e23dccbcb62","Type":"ContainerStarted","Data":"f0425b40a39261a6a3758a82adffa5c3ef103fb1a499c7ed8eff95dd7c798761"}
Oct 30 22:00:01.139421 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:01.139402    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"]
Oct 30 22:00:01.187355 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:01.187329    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"]
Oct 30 22:00:01.226747 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:00:01.226708    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0561cf7c_914f_4660_8b42_ecb3c42bec98.slice/crio-1b9808d8ef3e81f86d9915f5eb8af220ce940d40b2066d8d76247e523d506a9e WatchSource:0}: Error finding container 1b9808d8ef3e81f86d9915f5eb8af220ce940d40b2066d8d76247e523d506a9e: Status 404 returned error can't find the container with id 1b9808d8ef3e81f86d9915f5eb8af220ce940d40b2066d8d76247e523d506a9e
Oct 30 22:00:02.148997 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.148959    5586 generic.go:356] "Generic (PLEG): container finished" podID="11ca645b-a522-40ff-bd25-c1ee49f02e58" containerID="69ba1d66289ca37a7fb609e9f845df8cd5ce857577efc89014b2a9002cefceb1" exitCode=60
Oct 30 22:00:02.149504 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.149045    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n" event={"ID":"11ca645b-a522-40ff-bd25-c1ee49f02e58","Type":"ContainerDied","Data":"69ba1d66289ca37a7fb609e9f845df8cd5ce857577efc89014b2a9002cefceb1"}
Oct 30 22:00:02.149504 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.149389    5586 scope.go:122] "RemoveContainer" containerID="69ba1d66289ca37a7fb609e9f845df8cd5ce857577efc89014b2a9002cefceb1"
Oct 30 22:00:02.150928 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.150902    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn" event={"ID":"78ef55a3-8cd6-4b4d-9231-0a9745440fc6","Type":"ContainerStarted","Data":"fcab005bd99ed2a7d46b34000763abfb09eeb1e6c571e57d57bd0abd2fc70b2a"}
Oct 30 22:00:02.151047 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.150954    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn" event={"ID":"78ef55a3-8cd6-4b4d-9231-0a9745440fc6","Type":"ContainerStarted","Data":"939ee9dbf09c5ecd3c92e5e0d812061386b958ff655896a90470f3b8a0ea1ecc"}
Oct 30 22:00:02.153096 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.153073    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq" event={"ID":"26b31903-7a3a-4f41-b955-764d32eafb50","Type":"ContainerStarted","Data":"7cba8320527b654bdd0a2163cd09bf572b6172f8bdb5d14e388b4608108254f4"}
Oct 30 22:00:02.153468 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.153441    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq" event={"ID":"26b31903-7a3a-4f41-b955-764d32eafb50","Type":"ContainerStarted","Data":"cafc4946ce9f0cbfbf702f69ee11608e629e37ea7cea18a8daee08b247e76249"}
Oct 30 22:00:02.154904 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.154882    5586 generic.go:356] "Generic (PLEG): container finished" podID="f2822b90-308a-4149-a838-3e23dccbcb62" containerID="509803c2172826e46ee7e28ec180ec4905eea85b6256e349aefaecae602ea0a4" exitCode=60
Oct 30 22:00:02.154995 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.154962    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2" event={"ID":"f2822b90-308a-4149-a838-3e23dccbcb62","Type":"ContainerDied","Data":"509803c2172826e46ee7e28ec180ec4905eea85b6256e349aefaecae602ea0a4"}
Oct 30 22:00:02.155266 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.155209    5586 scope.go:122] "RemoveContainer" containerID="509803c2172826e46ee7e28ec180ec4905eea85b6256e349aefaecae602ea0a4"
Oct 30 22:00:02.156749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.156715    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6" event={"ID":"0561cf7c-914f-4660-8b42-ecb3c42bec98","Type":"ContainerStarted","Data":"160c0fd191d704f1e7b655b7fda10b6c938f96c02a8da1171b33880113c2d293"}
Oct 30 22:00:02.156870 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.156759    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6" event={"ID":"0561cf7c-914f-4660-8b42-ecb3c42bec98","Type":"ContainerStarted","Data":"1b9808d8ef3e81f86d9915f5eb8af220ce940d40b2066d8d76247e523d506a9e"}
Oct 30 22:00:02.399038 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.398975    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq" podStartSLOduration=2.398960675 podStartE2EDuration="2.398960675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:02.394852914 +0000 UTC m=+1545.066756850" watchObservedRunningTime="2026-10-30 22:00:02.398960675 +0000 UTC m=+1545.070864610"
Oct 30 22:00:02.640441 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.640342    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn" podStartSLOduration=2.64032774 podStartE2EDuration="2.64032774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:02.637294693 +0000 UTC m=+1545.309198627" watchObservedRunningTime="2026-10-30 22:00:02.64032774 +0000 UTC m=+1545.312231675"
Oct 30 22:00:02.766652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:02.766592    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6" podStartSLOduration=2.766576199 podStartE2EDuration="2.766576199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:02.762325449 +0000 UTC m=+1545.434229383" watchObservedRunningTime="2026-10-30 22:00:02.766576199 +0000 UTC m=+1545.438480133"
Oct 30 22:00:03.171639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:03.171593    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2" event={"ID":"f2822b90-308a-4149-a838-3e23dccbcb62","Type":"ContainerStarted","Data":"c4b0967211eb1093e72e94b3ccb97aef302821fdf87da0b6223887631692f876"}
Oct 30 22:00:03.173489 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:03.173461    5586 generic.go:356] "Generic (PLEG): container finished" podID="0561cf7c-914f-4660-8b42-ecb3c42bec98" containerID="160c0fd191d704f1e7b655b7fda10b6c938f96c02a8da1171b33880113c2d293" exitCode=60
Oct 30 22:00:03.173659 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:03.173548    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6" event={"ID":"0561cf7c-914f-4660-8b42-ecb3c42bec98","Type":"ContainerDied","Data":"160c0fd191d704f1e7b655b7fda10b6c938f96c02a8da1171b33880113c2d293"}
Oct 30 22:00:03.173856 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:03.173838    5586 scope.go:122] "RemoveContainer" containerID="160c0fd191d704f1e7b655b7fda10b6c938f96c02a8da1171b33880113c2d293"
Oct 30 22:00:03.175989 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:03.175965    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n" event={"ID":"11ca645b-a522-40ff-bd25-c1ee49f02e58","Type":"ContainerStarted","Data":"89ec189388d8fddfea9d0f88d019c75f293b87edf73f742476bdfbd2c27534c4"}
Oct 30 22:00:03.177993 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:03.177803    5586 generic.go:356] "Generic (PLEG): container finished" podID="78ef55a3-8cd6-4b4d-9231-0a9745440fc6" containerID="fcab005bd99ed2a7d46b34000763abfb09eeb1e6c571e57d57bd0abd2fc70b2a" exitCode=60
Oct 30 22:00:03.177993 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:03.177886    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn" event={"ID":"78ef55a3-8cd6-4b4d-9231-0a9745440fc6","Type":"ContainerDied","Data":"fcab005bd99ed2a7d46b34000763abfb09eeb1e6c571e57d57bd0abd2fc70b2a"}
Oct 30 22:00:03.178130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:03.178115    5586 scope.go:122] "RemoveContainer" containerID="fcab005bd99ed2a7d46b34000763abfb09eeb1e6c571e57d57bd0abd2fc70b2a"
Oct 30 22:00:03.179603 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:03.179579    5586 generic.go:356] "Generic (PLEG): container finished" podID="26b31903-7a3a-4f41-b955-764d32eafb50" containerID="7cba8320527b654bdd0a2163cd09bf572b6172f8bdb5d14e388b4608108254f4" exitCode=60
Oct 30 22:00:03.179672 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:03.179652    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq" event={"ID":"26b31903-7a3a-4f41-b955-764d32eafb50","Type":"ContainerDied","Data":"7cba8320527b654bdd0a2163cd09bf572b6172f8bdb5d14e388b4608108254f4"}
Oct 30 22:00:03.179847 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:03.179832    5586 scope.go:122] "RemoveContainer" containerID="7cba8320527b654bdd0a2163cd09bf572b6172f8bdb5d14e388b4608108254f4"
Oct 30 22:00:03.300065 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:03.300008    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2" podStartSLOduration=3.299988724 podStartE2EDuration="3.299988724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:03.295571359 +0000 UTC m=+1545.967475294" watchObservedRunningTime="2026-10-30 22:00:03.299988724 +0000 UTC m=+1545.971892661"
Oct 30 22:00:03.541183 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:03.541134    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n" podStartSLOduration=3.5411170370000002 podStartE2EDuration="3.541117037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:03.53736819 +0000 UTC m=+1546.209272125" watchObservedRunningTime="2026-10-30 22:00:03.541117037 +0000 UTC m=+1546.213020972"
Oct 30 22:00:04.190682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:04.190632    5586 generic.go:356] "Generic (PLEG): container finished" podID="11ca645b-a522-40ff-bd25-c1ee49f02e58" containerID="89ec189388d8fddfea9d0f88d019c75f293b87edf73f742476bdfbd2c27534c4" exitCode=60
Oct 30 22:00:04.191194 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:04.190742    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n" event={"ID":"11ca645b-a522-40ff-bd25-c1ee49f02e58","Type":"ContainerDied","Data":"89ec189388d8fddfea9d0f88d019c75f293b87edf73f742476bdfbd2c27534c4"}
Oct 30 22:00:04.191194 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:04.190802    5586 scope.go:122] "RemoveContainer" containerID="69ba1d66289ca37a7fb609e9f845df8cd5ce857577efc89014b2a9002cefceb1"
Oct 30 22:00:04.191194 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:04.191179    5586 scope.go:122] "RemoveContainer" containerID="89ec189388d8fddfea9d0f88d019c75f293b87edf73f742476bdfbd2c27534c4"
Oct 30 22:00:04.191671 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:04.191635    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n_odh-ai-gateway-infra(11ca645b-a522-40ff-bd25-c1ee49f02e58)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n" podUID="11ca645b-a522-40ff-bd25-c1ee49f02e58"
Oct 30 22:00:04.193089 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:04.193066    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn" event={"ID":"78ef55a3-8cd6-4b4d-9231-0a9745440fc6","Type":"ContainerStarted","Data":"055abc85bd6c65a746564dba9971361bdd10172e4b91fbb75affd3ac88faa0ef"}
Oct 30 22:00:04.195370 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:04.195347    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq" event={"ID":"26b31903-7a3a-4f41-b955-764d32eafb50","Type":"ContainerStarted","Data":"cc9004308dbe2650ecd9b13d8a4948cd32c9f9f81b4c28cc3c04f90cf54faf7e"}
Oct 30 22:00:04.197657 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:04.197634    5586 generic.go:356] "Generic (PLEG): container finished" podID="f2822b90-308a-4149-a838-3e23dccbcb62" containerID="c4b0967211eb1093e72e94b3ccb97aef302821fdf87da0b6223887631692f876" exitCode=60
Oct 30 22:00:04.197760 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:04.197726    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2" event={"ID":"f2822b90-308a-4149-a838-3e23dccbcb62","Type":"ContainerDied","Data":"c4b0967211eb1093e72e94b3ccb97aef302821fdf87da0b6223887631692f876"}
Oct 30 22:00:04.197989 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:04.197971    5586 scope.go:122] "RemoveContainer" containerID="c4b0967211eb1093e72e94b3ccb97aef302821fdf87da0b6223887631692f876"
Oct 30 22:00:04.198473 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:04.198330    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2_odh-ai-gateway-infra(f2822b90-308a-4149-a838-3e23dccbcb62)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2" podUID="f2822b90-308a-4149-a838-3e23dccbcb62"
Oct 30 22:00:04.200165 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:04.200140    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6" event={"ID":"0561cf7c-914f-4660-8b42-ecb3c42bec98","Type":"ContainerStarted","Data":"ee66b2b985d6c068719a1642a64cf66082ab1d11cbcac7a7eb594c54a41dc070"}
Oct 30 22:00:04.206168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:04.206147    5586 scope.go:122] "RemoveContainer" containerID="509803c2172826e46ee7e28ec180ec4905eea85b6256e349aefaecae602ea0a4"
Oct 30 22:00:05.210929 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.210892    5586 scope.go:122] "RemoveContainer" containerID="c4b0967211eb1093e72e94b3ccb97aef302821fdf87da0b6223887631692f876"
Oct 30 22:00:05.211441 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:05.211372    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2_odh-ai-gateway-infra(f2822b90-308a-4149-a838-3e23dccbcb62)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2" podUID="f2822b90-308a-4149-a838-3e23dccbcb62"
Oct 30 22:00:05.213366 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.213335    5586 generic.go:356] "Generic (PLEG): container finished" podID="0561cf7c-914f-4660-8b42-ecb3c42bec98" containerID="ee66b2b985d6c068719a1642a64cf66082ab1d11cbcac7a7eb594c54a41dc070" exitCode=60
Oct 30 22:00:05.214011 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.213531    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6" event={"ID":"0561cf7c-914f-4660-8b42-ecb3c42bec98","Type":"ContainerDied","Data":"ee66b2b985d6c068719a1642a64cf66082ab1d11cbcac7a7eb594c54a41dc070"}
Oct 30 22:00:05.214138 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.213904    5586 scope.go:122] "RemoveContainer" containerID="ee66b2b985d6c068719a1642a64cf66082ab1d11cbcac7a7eb594c54a41dc070"
Oct 30 22:00:05.214372 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.214182    5586 scope.go:122] "RemoveContainer" containerID="160c0fd191d704f1e7b655b7fda10b6c938f96c02a8da1171b33880113c2d293"
Oct 30 22:00:05.214525 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:05.214491    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6_odh-ai-gateway-infra(0561cf7c-914f-4660-8b42-ecb3c42bec98)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6" podUID="0561cf7c-914f-4660-8b42-ecb3c42bec98"
Oct 30 22:00:05.216670 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.216653    5586 scope.go:122] "RemoveContainer" containerID="89ec189388d8fddfea9d0f88d019c75f293b87edf73f742476bdfbd2c27534c4"
Oct 30 22:00:05.216979 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:05.216951    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n_odh-ai-gateway-infra(11ca645b-a522-40ff-bd25-c1ee49f02e58)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n" podUID="11ca645b-a522-40ff-bd25-c1ee49f02e58"
Oct 30 22:00:05.218305 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.218282    5586 generic.go:356] "Generic (PLEG): container finished" podID="78ef55a3-8cd6-4b4d-9231-0a9745440fc6" containerID="055abc85bd6c65a746564dba9971361bdd10172e4b91fbb75affd3ac88faa0ef" exitCode=60
Oct 30 22:00:05.218396 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.218340    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn" event={"ID":"78ef55a3-8cd6-4b4d-9231-0a9745440fc6","Type":"ContainerDied","Data":"055abc85bd6c65a746564dba9971361bdd10172e4b91fbb75affd3ac88faa0ef"}
Oct 30 22:00:05.218609 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.218595    5586 scope.go:122] "RemoveContainer" containerID="055abc85bd6c65a746564dba9971361bdd10172e4b91fbb75affd3ac88faa0ef"
Oct 30 22:00:05.218861 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:05.218841    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn_odh-ai-gateway-infra(78ef55a3-8cd6-4b4d-9231-0a9745440fc6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn" podUID="78ef55a3-8cd6-4b4d-9231-0a9745440fc6"
Oct 30 22:00:05.220537 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.220513    5586 generic.go:356] "Generic (PLEG): container finished" podID="26b31903-7a3a-4f41-b955-764d32eafb50" containerID="cc9004308dbe2650ecd9b13d8a4948cd32c9f9f81b4c28cc3c04f90cf54faf7e" exitCode=60
Oct 30 22:00:05.220640 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.220590    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq" event={"ID":"26b31903-7a3a-4f41-b955-764d32eafb50","Type":"ContainerDied","Data":"cc9004308dbe2650ecd9b13d8a4948cd32c9f9f81b4c28cc3c04f90cf54faf7e"}
Oct 30 22:00:05.220779 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.220765    5586 scope.go:122] "RemoveContainer" containerID="cc9004308dbe2650ecd9b13d8a4948cd32c9f9f81b4c28cc3c04f90cf54faf7e"
Oct 30 22:00:05.221071 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:05.221048    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889960-gq2xq_odh-ai-gateway-infra(26b31903-7a3a-4f41-b955-764d32eafb50)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq" podUID="26b31903-7a3a-4f41-b955-764d32eafb50"
Oct 30 22:00:05.228171 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.228149    5586 scope.go:122] "RemoveContainer" containerID="fcab005bd99ed2a7d46b34000763abfb09eeb1e6c571e57d57bd0abd2fc70b2a"
Oct 30 22:00:05.242922 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.242895    5586 scope.go:122] "RemoveContainer" containerID="7cba8320527b654bdd0a2163cd09bf572b6172f8bdb5d14e388b4608108254f4"
Oct 30 22:00:05.876225 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:05.876181    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:06.235162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:06.235127    5586 scope.go:122] "RemoveContainer" containerID="055abc85bd6c65a746564dba9971361bdd10172e4b91fbb75affd3ac88faa0ef"
Oct 30 22:00:06.235713 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:06.235615    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn_odh-ai-gateway-infra(78ef55a3-8cd6-4b4d-9231-0a9745440fc6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn" podUID="78ef55a3-8cd6-4b4d-9231-0a9745440fc6"
Oct 30 22:00:06.237099 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:06.237077    5586 scope.go:122] "RemoveContainer" containerID="cc9004308dbe2650ecd9b13d8a4948cd32c9f9f81b4c28cc3c04f90cf54faf7e"
Oct 30 22:00:06.237476 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:06.237444    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889960-gq2xq_odh-ai-gateway-infra(26b31903-7a3a-4f41-b955-764d32eafb50)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq" podUID="26b31903-7a3a-4f41-b955-764d32eafb50"
Oct 30 22:00:06.238890 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:06.238872    5586 scope.go:122] "RemoveContainer" containerID="ee66b2b985d6c068719a1642a64cf66082ab1d11cbcac7a7eb594c54a41dc070"
Oct 30 22:00:06.239087 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:06.239070    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6_odh-ai-gateway-infra(0561cf7c-914f-4660-8b42-ecb3c42bec98)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6" podUID="0561cf7c-914f-4660-8b42-ecb3c42bec98"
Oct 30 22:00:07.930722 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:07.930688    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:08.240854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:08.240752    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:08.474913 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:08.474877    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:08.996889 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:08.996851    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:09.208258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:09.208157    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:09.679158 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:09.679120    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:09.888806 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:09.888756    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:11.107132 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:11.107093    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:11.338515 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:11.338468    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:13.643699 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:13.643659    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:13.702773 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:13.702733    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"]
Oct 30 22:00:13.950986 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:13.950957    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"
Oct 30 22:00:14.057278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:14.056457    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78ef55a3-8cd6-4b4d-9231-0a9745440fc6-kube-api-access-c4sb5\" (UniqueName: \"kubernetes.io/projected/78ef55a3-8cd6-4b4d-9231-0a9745440fc6-kube-api-access-c4sb5\") pod \"78ef55a3-8cd6-4b4d-9231-0a9745440fc6\" (UID: \"78ef55a3-8cd6-4b4d-9231-0a9745440fc6\") "
Oct 30 22:00:14.057278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:14.056726    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78ef55a3-8cd6-4b4d-9231-0a9745440fc6-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78ef55a3-8cd6-4b4d-9231-0a9745440fc6-maas-api-service-ca\") pod \"78ef55a3-8cd6-4b4d-9231-0a9745440fc6\" (UID: \"78ef55a3-8cd6-4b4d-9231-0a9745440fc6\") "
Oct 30 22:00:14.058190 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:14.058155    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78ef55a3-8cd6-4b4d-9231-0a9745440fc6-maas-api-service-ca" pod "78ef55a3-8cd6-4b4d-9231-0a9745440fc6" (UID: "78ef55a3-8cd6-4b4d-9231-0a9745440fc6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:14.060820 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:14.060786    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78ef55a3-8cd6-4b4d-9231-0a9745440fc6-kube-api-access-c4sb5" pod "78ef55a3-8cd6-4b4d-9231-0a9745440fc6" (UID: "78ef55a3-8cd6-4b4d-9231-0a9745440fc6"). InnerVolumeSpecName "kube-api-access-c4sb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:14.158844 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:14.158805    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78ef55a3-8cd6-4b4d-9231-0a9745440fc6-maas-api-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.158844 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:14.158836    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4sb5\" (UniqueName: \"kubernetes.io/projected/78ef55a3-8cd6-4b4d-9231-0a9745440fc6-kube-api-access-c4sb5\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.416916 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:14.416881    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"
Oct 30 22:00:14.417100 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:14.416884    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn" event={"ID":"78ef55a3-8cd6-4b4d-9231-0a9745440fc6","Type":"ContainerDied","Data":"939ee9dbf09c5ecd3c92e5e0d812061386b958ff655896a90470f3b8a0ea1ecc"}
Oct 30 22:00:14.417100 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:14.417038    5586 scope.go:122] "RemoveContainer" containerID="055abc85bd6c65a746564dba9971361bdd10172e4b91fbb75affd3ac88faa0ef"
Oct 30 22:00:14.672320 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:14.670902    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"]
Oct 30 22:00:14.683431 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:14.682222    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-be51942b-29889960-zqhhn"]
Oct 30 22:00:15.432156 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:15.432120    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:15.450114 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:15.450075    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78ef55a3-8cd6-4b4d-9231-0a9745440fc6" path="/var/lib/kubelet/pods/78ef55a3-8cd6-4b4d-9231-0a9745440fc6/volumes"
Oct 30 22:00:15.706905 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:15.706814    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:15.860791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:15.860742    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:16.440318 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:16.440293    5586 scope.go:122] "RemoveContainer" containerID="c4b0967211eb1093e72e94b3ccb97aef302821fdf87da0b6223887631692f876"
Oct 30 22:00:16.440524 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:16.440490    5586 scope.go:122] "RemoveContainer" containerID="89ec189388d8fddfea9d0f88d019c75f293b87edf73f742476bdfbd2c27534c4"
Oct 30 22:00:16.691336 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:16.691205    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:17.354852 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:17.354821    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:17.455689 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:17.455645    5586 generic.go:356] "Generic (PLEG): container finished" podID="11ca645b-a522-40ff-bd25-c1ee49f02e58" containerID="d1e5202425fdaa4a8878d33f4ec3db4bd29c837154f1723d2504aeed50fd74b2" exitCode=60
Oct 30 22:00:17.459973 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:17.459948    5586 scope.go:122] "RemoveContainer" containerID="ee66b2b985d6c068719a1642a64cf66082ab1d11cbcac7a7eb594c54a41dc070"
Oct 30 22:00:17.460134 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:17.460020    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n" event={"ID":"11ca645b-a522-40ff-bd25-c1ee49f02e58","Type":"ContainerDied","Data":"d1e5202425fdaa4a8878d33f4ec3db4bd29c837154f1723d2504aeed50fd74b2"}
Oct 30 22:00:17.460134 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:17.460065    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2" event={"ID":"f2822b90-308a-4149-a838-3e23dccbcb62","Type":"ContainerStarted","Data":"e64829a2013de0c6f377750447af503c2fccb635f5da4e821affa271a41bc0ad"}
Oct 30 22:00:17.460134 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:17.460121    5586 scope.go:122] "RemoveContainer" containerID="89ec189388d8fddfea9d0f88d019c75f293b87edf73f742476bdfbd2c27534c4"
Oct 30 22:00:17.461779 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:17.460731    5586 scope.go:122] "RemoveContainer" containerID="d1e5202425fdaa4a8878d33f4ec3db4bd29c837154f1723d2504aeed50fd74b2"
Oct 30 22:00:17.461779 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:17.461052    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n_odh-ai-gateway-infra(11ca645b-a522-40ff-bd25-c1ee49f02e58)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n" podUID="11ca645b-a522-40ff-bd25-c1ee49f02e58"
Oct 30 22:00:18.365190 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:18.365102    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:00:18.440027 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.440000    5586 scope.go:122] "RemoveContainer" containerID="cc9004308dbe2650ecd9b13d8a4948cd32c9f9f81b4c28cc3c04f90cf54faf7e"
Oct 30 22:00:18.472297 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.472266    5586 generic.go:356] "Generic (PLEG): container finished" podID="f2822b90-308a-4149-a838-3e23dccbcb62" containerID="e64829a2013de0c6f377750447af503c2fccb635f5da4e821affa271a41bc0ad" exitCode=60
Oct 30 22:00:18.472443 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.472319    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2" event={"ID":"f2822b90-308a-4149-a838-3e23dccbcb62","Type":"ContainerDied","Data":"e64829a2013de0c6f377750447af503c2fccb635f5da4e821affa271a41bc0ad"}
Oct 30 22:00:18.472443 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.472390    5586 scope.go:122] "RemoveContainer" containerID="c4b0967211eb1093e72e94b3ccb97aef302821fdf87da0b6223887631692f876"
Oct 30 22:00:18.472855 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.472834    5586 scope.go:122] "RemoveContainer" containerID="e64829a2013de0c6f377750447af503c2fccb635f5da4e821affa271a41bc0ad"
Oct 30 22:00:18.473220 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:18.473202    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2_odh-ai-gateway-infra(f2822b90-308a-4149-a838-3e23dccbcb62)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2" podUID="f2822b90-308a-4149-a838-3e23dccbcb62"
Oct 30 22:00:18.475192 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.475164    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6" event={"ID":"0561cf7c-914f-4660-8b42-ecb3c42bec98","Type":"ContainerStarted","Data":"a257952a0786be232a0e738d64664d1a2ce468ed3558b4934b6e74d1afb40117"}
Oct 30 22:00:18.614858 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.614821    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"]
Oct 30 22:00:18.733095 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.733041    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"]
Oct 30 22:00:18.851283 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.848421    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"
Oct 30 22:00:18.911620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.911568    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:18.974728 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.974699    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11ca645b-a522-40ff-bd25-c1ee49f02e58-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11ca645b-a522-40ff-bd25-c1ee49f02e58-maas-api-service-ca\") pod \"11ca645b-a522-40ff-bd25-c1ee49f02e58\" (UID: \"11ca645b-a522-40ff-bd25-c1ee49f02e58\") "
Oct 30 22:00:18.974896 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.974876    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11ca645b-a522-40ff-bd25-c1ee49f02e58-kube-api-access-8kzfs\" (UniqueName: \"kubernetes.io/projected/11ca645b-a522-40ff-bd25-c1ee49f02e58-kube-api-access-8kzfs\") pod \"11ca645b-a522-40ff-bd25-c1ee49f02e58\" (UID: \"11ca645b-a522-40ff-bd25-c1ee49f02e58\") "
Oct 30 22:00:18.975308 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.975280    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11ca645b-a522-40ff-bd25-c1ee49f02e58-maas-api-service-ca" pod "11ca645b-a522-40ff-bd25-c1ee49f02e58" (UID: "11ca645b-a522-40ff-bd25-c1ee49f02e58"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:18.975823 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.975800    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11ca645b-a522-40ff-bd25-c1ee49f02e58-maas-api-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:18.977162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:18.977132    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11ca645b-a522-40ff-bd25-c1ee49f02e58-kube-api-access-8kzfs" pod "11ca645b-a522-40ff-bd25-c1ee49f02e58" (UID: "11ca645b-a522-40ff-bd25-c1ee49f02e58"). InnerVolumeSpecName "kube-api-access-8kzfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:19.082201 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.082161    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kzfs\" (UniqueName: \"kubernetes.io/projected/11ca645b-a522-40ff-bd25-c1ee49f02e58-kube-api-access-8kzfs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:19.141583 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.141551    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:19.495773 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.495728    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n" event={"ID":"11ca645b-a522-40ff-bd25-c1ee49f02e58","Type":"ContainerDied","Data":"69ef1ce83df2f48b64390724b3c59fa7b7365dd6a0b9491ee439674c732a9ec8"}
Oct 30 22:00:19.495773 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.495755    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"
Oct 30 22:00:19.496342 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.495841    5586 scope.go:122] "RemoveContainer" containerID="d1e5202425fdaa4a8878d33f4ec3db4bd29c837154f1723d2504aeed50fd74b2"
Oct 30 22:00:19.500276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.500202    5586 generic.go:356] "Generic (PLEG): container finished" podID="26b31903-7a3a-4f41-b955-764d32eafb50" containerID="3377babacff26772c1810f7125c157180f228a3df7ddb6aa7a53d825760cc008" exitCode=60
Oct 30 22:00:19.500403 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.500311    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq" event={"ID":"26b31903-7a3a-4f41-b955-764d32eafb50","Type":"ContainerDied","Data":"3377babacff26772c1810f7125c157180f228a3df7ddb6aa7a53d825760cc008"}
Oct 30 22:00:19.500900 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.500875    5586 scope.go:122] "RemoveContainer" containerID="3377babacff26772c1810f7125c157180f228a3df7ddb6aa7a53d825760cc008"
Oct 30 22:00:19.501383 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:19.501356    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889960-gq2xq_odh-ai-gateway-infra(26b31903-7a3a-4f41-b955-764d32eafb50)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq" podUID="26b31903-7a3a-4f41-b955-764d32eafb50"
Oct 30 22:00:19.504800 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.504776    5586 generic.go:356] "Generic (PLEG): container finished" podID="0561cf7c-914f-4660-8b42-ecb3c42bec98" containerID="a257952a0786be232a0e738d64664d1a2ce468ed3558b4934b6e74d1afb40117" exitCode=60
Oct 30 22:00:19.504898 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.504850    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6" event={"ID":"0561cf7c-914f-4660-8b42-ecb3c42bec98","Type":"ContainerDied","Data":"a257952a0786be232a0e738d64664d1a2ce468ed3558b4934b6e74d1afb40117"}
Oct 30 22:00:19.505328 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.505311    5586 scope.go:122] "RemoveContainer" containerID="a257952a0786be232a0e738d64664d1a2ce468ed3558b4934b6e74d1afb40117"
Oct 30 22:00:19.505723 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:19.505700    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6_odh-ai-gateway-infra(0561cf7c-914f-4660-8b42-ecb3c42bec98)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6" podUID="0561cf7c-914f-4660-8b42-ecb3c42bec98"
Oct 30 22:00:19.515574 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.515548    5586 scope.go:122] "RemoveContainer" containerID="cc9004308dbe2650ecd9b13d8a4948cd32c9f9f81b4c28cc3c04f90cf54faf7e"
Oct 30 22:00:19.517427 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:19.517398    5586 kuberuntime_gc.go:392] "Failed to remove container log dead symlink" err="remove /var/log/containers/maas-api-key-cleanup-29889960-gq2xq_odh-ai-gateway-infra_cleanup-cc9004308dbe2650ecd9b13d8a4948cd32c9f9f81b4c28cc3c04f90cf54faf7e.log: no such file or directory" path="/var/log/containers/maas-api-key-cleanup-29889960-gq2xq_odh-ai-gateway-infra_cleanup-cc9004308dbe2650ecd9b13d8a4948cd32c9f9f81b4c28cc3c04f90cf54faf7e.log"
Oct 30 22:00:19.531880 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.531848    5586 scope.go:122] "RemoveContainer" containerID="ee66b2b985d6c068719a1642a64cf66082ab1d11cbcac7a7eb594c54a41dc070"
Oct 30 22:00:19.698661 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.698586    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"
Oct 30 22:00:19.737087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.737050    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"]
Oct 30 22:00:19.788189 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.788145    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"]
Oct 30 22:00:19.795302 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.795270    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-bc009f-29889960-6mp9n"]
Oct 30 22:00:19.801941 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.801909    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2822b90-308a-4149-a838-3e23dccbcb62-kube-api-access-fsvmt\" (UniqueName: \"kubernetes.io/projected/f2822b90-308a-4149-a838-3e23dccbcb62-kube-api-access-fsvmt\") pod \"f2822b90-308a-4149-a838-3e23dccbcb62\" (UID: \"f2822b90-308a-4149-a838-3e23dccbcb62\") "
Oct 30 22:00:19.802157 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.802136    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2822b90-308a-4149-a838-3e23dccbcb62-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2822b90-308a-4149-a838-3e23dccbcb62-maas-api-service-ca\") pod \"f2822b90-308a-4149-a838-3e23dccbcb62\" (UID: \"f2822b90-308a-4149-a838-3e23dccbcb62\") "
Oct 30 22:00:19.802528 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.802504    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2822b90-308a-4149-a838-3e23dccbcb62-maas-api-service-ca" pod "f2822b90-308a-4149-a838-3e23dccbcb62" (UID: "f2822b90-308a-4149-a838-3e23dccbcb62"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:19.802901 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.802883    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2822b90-308a-4149-a838-3e23dccbcb62-maas-api-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:19.804223 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.804195    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2822b90-308a-4149-a838-3e23dccbcb62-kube-api-access-fsvmt" pod "f2822b90-308a-4149-a838-3e23dccbcb62" (UID: "f2822b90-308a-4149-a838-3e23dccbcb62"). InnerVolumeSpecName "kube-api-access-fsvmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:19.905040 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:19.905003    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsvmt\" (UniqueName: \"kubernetes.io/projected/f2822b90-308a-4149-a838-3e23dccbcb62-kube-api-access-fsvmt\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:20.023107 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:20.023015    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:20.525911 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:20.525766    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2" event={"ID":"f2822b90-308a-4149-a838-3e23dccbcb62","Type":"ContainerDied","Data":"f0425b40a39261a6a3758a82adffa5c3ef103fb1a499c7ed8eff95dd7c798761"}
Oct 30 22:00:20.525911 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:20.525839    5586 scope.go:122] "RemoveContainer" containerID="e64829a2013de0c6f377750447af503c2fccb635f5da4e821affa271a41bc0ad"
Oct 30 22:00:20.526524 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:20.525938    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"
Oct 30 22:00:20.683184 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:20.683095    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"]
Oct 30 22:00:20.865802 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:20.865726    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"]
Oct 30 22:00:20.872509 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:20.872431    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-dda9f5-29889960-2n5c2"]
Oct 30 22:00:20.883048 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:20.883021    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"
Oct 30 22:00:20.958114 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:20.958083    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"
Oct 30 22:00:21.035207 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.033386    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26b31903-7a3a-4f41-b955-764d32eafb50-kube-api-access-8fg4g\" (UniqueName: \"kubernetes.io/projected/26b31903-7a3a-4f41-b955-764d32eafb50-kube-api-access-8fg4g\") pod \"26b31903-7a3a-4f41-b955-764d32eafb50\" (UID: \"26b31903-7a3a-4f41-b955-764d32eafb50\") "
Oct 30 22:00:21.035207 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.033441    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/26b31903-7a3a-4f41-b955-764d32eafb50-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b31903-7a3a-4f41-b955-764d32eafb50-maas-api-service-ca\") pod \"26b31903-7a3a-4f41-b955-764d32eafb50\" (UID: \"26b31903-7a3a-4f41-b955-764d32eafb50\") "
Oct 30 22:00:21.035207 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.033517    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0561cf7c-914f-4660-8b42-ecb3c42bec98-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0561cf7c-914f-4660-8b42-ecb3c42bec98-maas-api-service-ca\") pod \"0561cf7c-914f-4660-8b42-ecb3c42bec98\" (UID: \"0561cf7c-914f-4660-8b42-ecb3c42bec98\") "
Oct 30 22:00:21.035207 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.033562    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0561cf7c-914f-4660-8b42-ecb3c42bec98-kube-api-access-ct22r\" (UniqueName: \"kubernetes.io/projected/0561cf7c-914f-4660-8b42-ecb3c42bec98-kube-api-access-ct22r\") pod \"0561cf7c-914f-4660-8b42-ecb3c42bec98\" (UID: \"0561cf7c-914f-4660-8b42-ecb3c42bec98\") "
Oct 30 22:00:21.035973 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.035945    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26b31903-7a3a-4f41-b955-764d32eafb50-maas-api-service-ca" pod "26b31903-7a3a-4f41-b955-764d32eafb50" (UID: "26b31903-7a3a-4f41-b955-764d32eafb50"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:21.038874 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.038534    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0561cf7c-914f-4660-8b42-ecb3c42bec98-maas-api-service-ca" pod "0561cf7c-914f-4660-8b42-ecb3c42bec98" (UID: "0561cf7c-914f-4660-8b42-ecb3c42bec98"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:21.047102 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.047053    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0561cf7c-914f-4660-8b42-ecb3c42bec98-kube-api-access-ct22r" pod "0561cf7c-914f-4660-8b42-ecb3c42bec98" (UID: "0561cf7c-914f-4660-8b42-ecb3c42bec98"). InnerVolumeSpecName "kube-api-access-ct22r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:21.098256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.092845    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26b31903-7a3a-4f41-b955-764d32eafb50-kube-api-access-8fg4g" pod "26b31903-7a3a-4f41-b955-764d32eafb50" (UID: "26b31903-7a3a-4f41-b955-764d32eafb50"). InnerVolumeSpecName "kube-api-access-8fg4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:21.138510 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.137633    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:21.140626 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.140499    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fg4g\" (UniqueName: \"kubernetes.io/projected/26b31903-7a3a-4f41-b955-764d32eafb50-kube-api-access-8fg4g\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:21.140626 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.140534    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b31903-7a3a-4f41-b955-764d32eafb50-maas-api-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:21.140626 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.140552    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0561cf7c-914f-4660-8b42-ecb3c42bec98-maas-api-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:21.140626 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.140567    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ct22r\" (UniqueName: \"kubernetes.io/projected/0561cf7c-914f-4660-8b42-ecb3c42bec98-kube-api-access-ct22r\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:21.459378 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.459335    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:21.462898 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.462859    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11ca645b-a522-40ff-bd25-c1ee49f02e58" path="/var/lib/kubelet/pods/11ca645b-a522-40ff-bd25-c1ee49f02e58/volumes"
Oct 30 22:00:21.463621 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.463602    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2822b90-308a-4149-a838-3e23dccbcb62" path="/var/lib/kubelet/pods/f2822b90-308a-4149-a838-3e23dccbcb62/volumes"
Oct 30 22:00:21.563041 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.563011    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"
Oct 30 22:00:21.563614 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.563033    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6" event={"ID":"0561cf7c-914f-4660-8b42-ecb3c42bec98","Type":"ContainerDied","Data":"1b9808d8ef3e81f86d9915f5eb8af220ce940d40b2066d8d76247e523d506a9e"}
Oct 30 22:00:21.563614 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.563094    5586 scope.go:122] "RemoveContainer" containerID="a257952a0786be232a0e738d64664d1a2ce468ed3558b4934b6e74d1afb40117"
Oct 30 22:00:21.564799 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.564770    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq" event={"ID":"26b31903-7a3a-4f41-b955-764d32eafb50","Type":"ContainerDied","Data":"cafc4946ce9f0cbfbf702f69ee11608e629e37ea7cea18a8daee08b247e76249"}
Oct 30 22:00:21.564912 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.564824    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"
Oct 30 22:00:21.575102 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.575081    5586 scope.go:122] "RemoveContainer" containerID="3377babacff26772c1810f7125c157180f228a3df7ddb6aa7a53d825760cc008"
Oct 30 22:00:21.752335 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.746066    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"]
Oct 30 22:00:21.752335 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.747477    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:21.757774 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.757730    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889960-gq2xq"]
Oct 30 22:00:21.883379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.883216    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"]
Oct 30 22:00:21.890767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:21.890730    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-9b51e1-29889960-q8vl6"]
Oct 30 22:00:23.031581 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:23.031546    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:23.246357 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:23.246307    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:23.449656 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:23.449625    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0561cf7c-914f-4660-8b42-ecb3c42bec98" path="/var/lib/kubelet/pods/0561cf7c-914f-4660-8b42-ecb3c42bec98/volumes"
Oct 30 22:00:23.450161 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:23.450144    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="26b31903-7a3a-4f41-b955-764d32eafb50" path="/var/lib/kubelet/pods/26b31903-7a3a-4f41-b955-764d32eafb50/volumes"
Oct 30 22:00:24.536528 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:24.536492    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:26.391359 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:26.391322    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:26.669411 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:26.669368    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:26.928212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:26.928106    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:27.180320 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:27.166896    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:28.067167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:28.067133    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"]
Oct 30 22:00:28.115545 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:28.115500    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b48a3910-ppk8r"]
Oct 30 22:00:28.728227 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:28.728189    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:28.903753 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:28.903670    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:29.238468 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:29.238426    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:29.458141 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:29.458099    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aef30478-1b5f-4cae-b205-ef1364298532" path="/var/lib/kubelet/pods/aef30478-1b5f-4cae-b205-ef1364298532/volumes"
Oct 30 22:00:29.596909 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:29.596816    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:29.861585 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:29.861480    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:30.748071 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:30.748037    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"]
Oct 30 22:00:30.748488 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:30.748268    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" podUID="fc730edc-8621-499c-a05a-22dc848e2727" containerName="istio-proxy" containerID="cri-o://186ccdae1e3b06c02e1d200aa92dd30a79aaf1a759aff1d3842b4e86fb484832" gracePeriod=30
Oct 30 22:00:30.816182 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:30.816148    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:31.744149 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:31.744107    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:32.003264 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:32.003152    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:34.697202 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:34.697035    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:36.031649 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.031581    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 22:00:36.050652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.050616    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-workload-socket\") pod \"fc730edc-8621-499c-a05a-22dc848e2727\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") "
Oct 30 22:00:36.050792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.050687    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc730edc-8621-499c-a05a-22dc848e2727-kube-api-access-qrpxf\" (UniqueName: \"kubernetes.io/projected/fc730edc-8621-499c-a05a-22dc848e2727-kube-api-access-qrpxf\") pod \"fc730edc-8621-499c-a05a-22dc848e2727\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") "
Oct 30 22:00:36.050792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.050744    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc730edc-8621-499c-a05a-22dc848e2727-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc730edc-8621-499c-a05a-22dc848e2727-istiod-ca-cert\") pod \"fc730edc-8621-499c-a05a-22dc848e2727\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") "
Oct 30 22:00:36.050957 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.050862    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-workload-certs\") pod \"fc730edc-8621-499c-a05a-22dc848e2727\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") "
Oct 30 22:00:36.050957 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.050924    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-workload-socket" pod "fc730edc-8621-499c-a05a-22dc848e2727" (UID: "fc730edc-8621-499c-a05a-22dc848e2727"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:36.051067 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.051008    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-istio-data\") pod \"fc730edc-8621-499c-a05a-22dc848e2727\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") "
Oct 30 22:00:36.051122 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.051068    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fc730edc-8621-499c-a05a-22dc848e2727-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc730edc-8621-499c-a05a-22dc848e2727-istio-podinfo\") pod \"fc730edc-8621-499c-a05a-22dc848e2727\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") "
Oct 30 22:00:36.051122 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.051075    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-workload-certs" pod "fc730edc-8621-499c-a05a-22dc848e2727" (UID: "fc730edc-8621-499c-a05a-22dc848e2727"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:36.051122 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.051093    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc730edc-8621-499c-a05a-22dc848e2727-istiod-ca-cert" pod "fc730edc-8621-499c-a05a-22dc848e2727" (UID: "fc730edc-8621-499c-a05a-22dc848e2727"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:36.051294 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.051173    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-credential-socket\") pod \"fc730edc-8621-499c-a05a-22dc848e2727\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") "
Oct 30 22:00:36.051363 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.051316    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc730edc-8621-499c-a05a-22dc848e2727-istio-token\" (UniqueName: \"kubernetes.io/projected/fc730edc-8621-499c-a05a-22dc848e2727-istio-token\") pod \"fc730edc-8621-499c-a05a-22dc848e2727\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") "
Oct 30 22:00:36.051363 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.051248    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-istio-data" pod "fc730edc-8621-499c-a05a-22dc848e2727" (UID: "fc730edc-8621-499c-a05a-22dc848e2727"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:36.051363 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.051346    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-istio-envoy\") pod \"fc730edc-8621-499c-a05a-22dc848e2727\" (UID: \"fc730edc-8621-499c-a05a-22dc848e2727\") "
Oct 30 22:00:36.051512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.051450    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-credential-socket" pod "fc730edc-8621-499c-a05a-22dc848e2727" (UID: "fc730edc-8621-499c-a05a-22dc848e2727"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:36.052226 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.052199    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:36.052226 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.052246    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:36.052226 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.052262    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:36.052489 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.052277    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc730edc-8621-499c-a05a-22dc848e2727-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:36.052489 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.052295    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:36.053124 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.053100    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc730edc-8621-499c-a05a-22dc848e2727-kube-api-access-qrpxf" pod "fc730edc-8621-499c-a05a-22dc848e2727" (UID: "fc730edc-8621-499c-a05a-22dc848e2727"). InnerVolumeSpecName "kube-api-access-qrpxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:36.053212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.053146    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc730edc-8621-499c-a05a-22dc848e2727-istio-podinfo" pod "fc730edc-8621-499c-a05a-22dc848e2727" (UID: "fc730edc-8621-499c-a05a-22dc848e2727"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:00:36.053464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.053444    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc730edc-8621-499c-a05a-22dc848e2727-istio-token" pod "fc730edc-8621-499c-a05a-22dc848e2727" (UID: "fc730edc-8621-499c-a05a-22dc848e2727"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:36.053527 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.053461    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-istio-envoy" pod "fc730edc-8621-499c-a05a-22dc848e2727" (UID: "fc730edc-8621-499c-a05a-22dc848e2727"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:36.153652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.153615    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc730edc-8621-499c-a05a-22dc848e2727-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:36.153652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.153650    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc730edc-8621-499c-a05a-22dc848e2727-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:36.153898 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.153664    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc730edc-8621-499c-a05a-22dc848e2727-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:36.153898 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.153676    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrpxf\" (UniqueName: \"kubernetes.io/projected/fc730edc-8621-499c-a05a-22dc848e2727-kube-api-access-qrpxf\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:36.721993 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.721957    5586 generic.go:356] "Generic (PLEG): container finished" podID="fc730edc-8621-499c-a05a-22dc848e2727" containerID="186ccdae1e3b06c02e1d200aa92dd30a79aaf1a759aff1d3842b4e86fb484832" exitCode=0
Oct 30 22:00:36.722174 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.722022    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" event={"ID":"fc730edc-8621-499c-a05a-22dc848e2727","Type":"ContainerDied","Data":"186ccdae1e3b06c02e1d200aa92dd30a79aaf1a759aff1d3842b4e86fb484832"}
Oct 30 22:00:36.722174 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.722026    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"
Oct 30 22:00:36.722174 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.722068    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9" event={"ID":"fc730edc-8621-499c-a05a-22dc848e2727","Type":"ContainerDied","Data":"3929289ec4bccbfc41040b94908e80ed2c473fea8d9c9fff3baa538e46d6a6c9"}
Oct 30 22:00:36.722174 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.722092    5586 scope.go:122] "RemoveContainer" containerID="186ccdae1e3b06c02e1d200aa92dd30a79aaf1a759aff1d3842b4e86fb484832"
Oct 30 22:00:36.734381 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.734360    5586 scope.go:122] "RemoveContainer" containerID="28ac2fcb43e22664271a4cf7e1bd5b9095f54aaf986de3e92293c61bc067b73e"
Oct 30 22:00:36.744791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.744766    5586 scope.go:122] "RemoveContainer" containerID="186ccdae1e3b06c02e1d200aa92dd30a79aaf1a759aff1d3842b4e86fb484832"
Oct 30 22:00:36.745080 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:36.745059    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"186ccdae1e3b06c02e1d200aa92dd30a79aaf1a759aff1d3842b4e86fb484832\": container with ID starting with 186ccdae1e3b06c02e1d200aa92dd30a79aaf1a759aff1d3842b4e86fb484832 not found: ID does not exist" containerID="186ccdae1e3b06c02e1d200aa92dd30a79aaf1a759aff1d3842b4e86fb484832"
Oct 30 22:00:36.745130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.745088    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"186ccdae1e3b06c02e1d200aa92dd30a79aaf1a759aff1d3842b4e86fb484832"} err="failed to get container status \"186ccdae1e3b06c02e1d200aa92dd30a79aaf1a759aff1d3842b4e86fb484832\": rpc error: code = NotFound desc = could not find container \"186ccdae1e3b06c02e1d200aa92dd30a79aaf1a759aff1d3842b4e86fb484832\": container with ID starting with 186ccdae1e3b06c02e1d200aa92dd30a79aaf1a759aff1d3842b4e86fb484832 not found: ID does not exist"
Oct 30 22:00:36.745130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.745106    5586 scope.go:122] "RemoveContainer" containerID="28ac2fcb43e22664271a4cf7e1bd5b9095f54aaf986de3e92293c61bc067b73e"
Oct 30 22:00:36.745392 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:36.745375    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28ac2fcb43e22664271a4cf7e1bd5b9095f54aaf986de3e92293c61bc067b73e\": container with ID starting with 28ac2fcb43e22664271a4cf7e1bd5b9095f54aaf986de3e92293c61bc067b73e not found: ID does not exist" containerID="28ac2fcb43e22664271a4cf7e1bd5b9095f54aaf986de3e92293c61bc067b73e"
Oct 30 22:00:36.745448 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.745397    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28ac2fcb43e22664271a4cf7e1bd5b9095f54aaf986de3e92293c61bc067b73e"} err="failed to get container status \"28ac2fcb43e22664271a4cf7e1bd5b9095f54aaf986de3e92293c61bc067b73e\": rpc error: code = NotFound desc = could not find container \"28ac2fcb43e22664271a4cf7e1bd5b9095f54aaf986de3e92293c61bc067b73e\": container with ID starting with 28ac2fcb43e22664271a4cf7e1bd5b9095f54aaf986de3e92293c61bc067b73e not found: ID does not exist"
Oct 30 22:00:36.950458 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.950425    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"]
Oct 30 22:00:36.957275 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:36.957228    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e585c53f-openshift-default-6b94747c57-7zfp9"]
Oct 30 22:00:37.449752 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:37.449717    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc730edc-8621-499c-a05a-22dc848e2727" path="/var/lib/kubelet/pods/fc730edc-8621-499c-a05a-22dc848e2727/volumes"
Oct 30 22:00:39.588374 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:39.588337    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:44.714449 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:44.714409    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:44.903256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:44.890872    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:48.366458 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:00:48.366412    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:00:48.530941 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:48.530888    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:49.504796 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:49.504756    5586 pod_container_manager_linux.go:218] "Failed to delete cgroup paths" cgroupName=["kubepods","burstable","pod11ca645b-a522-40ff-bd25-c1ee49f02e58"] err="unable to destroy cgroup paths for cgroup [kubepods burstable pod11ca645b-a522-40ff-bd25-c1ee49f02e58] : Timed out while waiting for systemd to remove kubepods-burstable-pod11ca645b_a522_40ff_bd25_c1ee49f02e58.slice"
Oct 30 22:00:49.504796 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:49.504792    5586 kubelet_pods.go:2797] "Failed to destroy orphaned pod cgroup" podUID="11ca645b-a522-40ff-bd25-c1ee49f02e58" err="failed to delete cgroup paths for [kubepods burstable pod11ca645b-a522-40ff-bd25-c1ee49f02e58] : unable to destroy cgroup paths for cgroup [kubepods burstable pod11ca645b-a522-40ff-bd25-c1ee49f02e58] : Timed out while waiting for systemd to remove kubepods-burstable-pod11ca645b_a522_40ff_bd25_c1ee49f02e58.slice"
Oct 30 22:00:49.920315 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:49.916281    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:52.442108 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:52.442069    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:53.723613 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:53.723568    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:54.052968 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:54.052878    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"]
Oct 30 22:00:54.102028 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:54.101991    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bf234ca6-5978x"]
Oct 30 22:00:54.719788 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:54.719742    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:54.942445 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:54.942404    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:55.452539 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:55.452493    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f787aa0a-0804-4d85-9af8-0d50161693fb" path="/var/lib/kubelet/pods/f787aa0a-0804-4d85-9af8-0d50161693fb/volumes"
Oct 30 22:00:59.682078 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:59.682035    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:00:59.854552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:00:59.854501    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:04.357097 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:04.357044    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:04.523494 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:04.523459    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:04.691905 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:04.691866    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:09.380840 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:09.380796    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:11.055207 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:11.055160    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"]
Oct 30 22:01:11.104188 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:11.104144    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-18505428-57hqw"]
Oct 30 22:01:11.449018 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:11.448986    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3235316-025d-4a9a-9315-7a80e65dcb66" path="/var/lib/kubelet/pods/d3235316-025d-4a9a-9315-7a80e65dcb66/volumes"
Oct 30 22:01:14.123161 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:14.123121    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:14.285656 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:14.285624    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:14.443606 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:14.443571    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:18.363939 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:01:18.363902    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:01:18.536962 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:18.536922    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:18.724703 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:18.724666    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:19.017032 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:19.016930    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:23.759548 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:23.759508    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:24.813173 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:24.813132    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:28.447141 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:28.446934    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:29.050497 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:29.050457    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"]
Oct 30 22:01:29.100285 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:29.100247    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ec238e1-gj8bb"]
Oct 30 22:01:29.495071 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:29.495029    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="661fbff3-6968-41c6-b8d1-ab9f2b22820a" path="/var/lib/kubelet/pods/661fbff3-6968-41c6-b8d1-ab9f2b22820a/volumes"
Oct 30 22:01:30.083737 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:30.083701    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:30.991255 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:30.991189    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"]
Oct 30 22:01:30.999671 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:30.999640    5586 cpu_manager.go:414] "containerMap: removing container" podUID="fc730edc-8621-499c-a05a-22dc848e2727" containerName="istio-proxy"
Oct 30 22:01:30.999890 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:30.999874    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc730edc-8621-499c-a05a-22dc848e2727" containerName="istio-proxy"
Oct 30 22:01:30.999983 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:30.999973    5586 cpu_manager.go:414] "containerMap: removing container" podUID="26b31903-7a3a-4f41-b955-764d32eafb50" containerName="cleanup"
Oct 30 22:01:31.000054 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.000044    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26b31903-7a3a-4f41-b955-764d32eafb50" containerName="cleanup"
Oct 30 22:01:31.000455 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.000438    5586 cpu_manager.go:414] "containerMap: removing container" podUID="f2822b90-308a-4149-a838-3e23dccbcb62" containerName="cleanup"
Oct 30 22:01:31.000617 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.000604    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2822b90-308a-4149-a838-3e23dccbcb62" containerName="cleanup"
Oct 30 22:01:31.000702 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.000692    5586 cpu_manager.go:414] "containerMap: removing container" podUID="0561cf7c-914f-4660-8b42-ecb3c42bec98" containerName="cleanup"
Oct 30 22:01:31.000779 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.000768    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0561cf7c-914f-4660-8b42-ecb3c42bec98" containerName="cleanup"
Oct 30 22:01:31.001144 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.000952    5586 cpu_manager.go:414] "containerMap: removing container" podUID="0561cf7c-914f-4660-8b42-ecb3c42bec98" containerName="cleanup"
Oct 30 22:01:31.001325 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.001306    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0561cf7c-914f-4660-8b42-ecb3c42bec98" containerName="cleanup"
Oct 30 22:01:31.001452 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.001440    5586 cpu_manager.go:414] "containerMap: removing container" podUID="11ca645b-a522-40ff-bd25-c1ee49f02e58" containerName="cleanup"
Oct 30 22:01:31.001531 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.001521    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11ca645b-a522-40ff-bd25-c1ee49f02e58" containerName="cleanup"
Oct 30 22:01:31.001638 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.001628    5586 cpu_manager.go:414] "containerMap: removing container" podUID="11ca645b-a522-40ff-bd25-c1ee49f02e58" containerName="cleanup"
Oct 30 22:01:31.001727 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.001716    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11ca645b-a522-40ff-bd25-c1ee49f02e58" containerName="cleanup"
Oct 30 22:01:31.001830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.001820    5586 cpu_manager.go:414] "containerMap: removing container" podUID="78ef55a3-8cd6-4b4d-9231-0a9745440fc6" containerName="cleanup"
Oct 30 22:01:31.001904 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.001894    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78ef55a3-8cd6-4b4d-9231-0a9745440fc6" containerName="cleanup"
Oct 30 22:01:31.001984 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.001975    5586 cpu_manager.go:414] "containerMap: removing container" podUID="26b31903-7a3a-4f41-b955-764d32eafb50" containerName="cleanup"
Oct 30 22:01:31.002056 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.002046    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26b31903-7a3a-4f41-b955-764d32eafb50" containerName="cleanup"
Oct 30 22:01:31.002138 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.002129    5586 cpu_manager.go:414] "containerMap: removing container" podUID="f2822b90-308a-4149-a838-3e23dccbcb62" containerName="cleanup"
Oct 30 22:01:31.002213 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.002202    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2822b90-308a-4149-a838-3e23dccbcb62" containerName="cleanup"
Oct 30 22:01:31.002312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.002299    5586 cpu_manager.go:414] "containerMap: removing container" podUID="26b31903-7a3a-4f41-b955-764d32eafb50" containerName="cleanup"
Oct 30 22:01:31.002393 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.002383    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26b31903-7a3a-4f41-b955-764d32eafb50" containerName="cleanup"
Oct 30 22:01:31.002486 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.002476    5586 cpu_manager.go:414] "containerMap: removing container" podUID="78ef55a3-8cd6-4b4d-9231-0a9745440fc6" containerName="cleanup"
Oct 30 22:01:31.002560 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.002550    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78ef55a3-8cd6-4b4d-9231-0a9745440fc6" containerName="cleanup"
Oct 30 22:01:31.002661 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.002650    5586 cpu_manager.go:414] "containerMap: removing container" podUID="11ca645b-a522-40ff-bd25-c1ee49f02e58" containerName="cleanup"
Oct 30 22:01:31.002736 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.002726    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11ca645b-a522-40ff-bd25-c1ee49f02e58" containerName="cleanup"
Oct 30 22:01:31.003021 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.003007    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="11ca645b-a522-40ff-bd25-c1ee49f02e58" containerName="cleanup"
Oct 30 22:01:31.003115 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.003105    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2822b90-308a-4149-a838-3e23dccbcb62" containerName="cleanup"
Oct 30 22:01:31.003200 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.003191    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="78ef55a3-8cd6-4b4d-9231-0a9745440fc6" containerName="cleanup"
Oct 30 22:01:31.003308 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.003298    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="26b31903-7a3a-4f41-b955-764d32eafb50" containerName="cleanup"
Oct 30 22:01:31.003392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.003383    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="26b31903-7a3a-4f41-b955-764d32eafb50" containerName="cleanup"
Oct 30 22:01:31.003469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.003460    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="0561cf7c-914f-4660-8b42-ecb3c42bec98" containerName="cleanup"
Oct 30 22:01:31.003543 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.003534    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="78ef55a3-8cd6-4b4d-9231-0a9745440fc6" containerName="cleanup"
Oct 30 22:01:31.003622 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.003613    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc730edc-8621-499c-a05a-22dc848e2727" containerName="istio-proxy"
Oct 30 22:01:31.003698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.003689    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc730edc-8621-499c-a05a-22dc848e2727" containerName="istio-proxy"
Oct 30 22:01:31.003781 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.003772    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2822b90-308a-4149-a838-3e23dccbcb62" containerName="cleanup"
Oct 30 22:01:31.003856 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.003847    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="0561cf7c-914f-4660-8b42-ecb3c42bec98" containerName="cleanup"
Oct 30 22:01:31.003930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.003921    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="0561cf7c-914f-4660-8b42-ecb3c42bec98" containerName="cleanup"
Oct 30 22:01:31.004007 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.003998    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="11ca645b-a522-40ff-bd25-c1ee49f02e58" containerName="cleanup"
Oct 30 22:01:31.023322 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.023273    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.112560 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.112511    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-data\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.155681 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.150468    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e2deec91-openshift-default-dockercfg-fhbgh\""
Oct 30 22:01:31.156025 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.155983    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08f613b9-8231-4e10-8a70-5dee789fc9f7-istiod-ca-cert\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.156266 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.156250    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-podinfo\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.156521 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.156491    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn26f\" (UniqueName: \"kubernetes.io/projected/08f613b9-8231-4e10-8a70-5dee789fc9f7-kube-api-access-hn26f\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.156661 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.156647    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-token\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.156810 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.156767    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-workload-certs\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.157153 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.157094    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-envoy\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.157310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.157296    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-workload-socket\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.157530 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.157516    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-credential-socket\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.163374 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.163270    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:31.254911 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.245320    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"]
Oct 30 22:01:31.317289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.291444    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-podinfo\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.317289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.291568    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn26f\" (UniqueName: \"kubernetes.io/projected/08f613b9-8231-4e10-8a70-5dee789fc9f7-kube-api-access-hn26f\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.317289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.291615    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-token\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.317289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.291649    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-workload-certs\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.317289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.291782    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-envoy\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.317289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.291819    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-workload-socket\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.317289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.291879    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-credential-socket\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.317289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.304426    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-data\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.317289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.304481    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08f613b9-8231-4e10-8a70-5dee789fc9f7-istiod-ca-cert\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.329138 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.328724    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08f613b9-8231-4e10-8a70-5dee789fc9f7-istiod-ca-cert\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.372130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.372096    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-workload-certs\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.394445 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.394410    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-credential-socket\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.407749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.407714    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-workload-socket\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.408597 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.408571    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-data\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.581250 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.580795    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-token\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.664258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.660118    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-podinfo\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.713259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.710695    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-envoy\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.713259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.711553    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn26f\" (UniqueName: \"kubernetes.io/projected/08f613b9-8231-4e10-8a70-5dee789fc9f7-kube-api-access-hn26f\") pod \"e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") " pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:31.969744 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:31.969696    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:32.082759 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:32.080041    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:33.438170 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:33.432629    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"]
Oct 30 22:01:33.445746 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:33.445712    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:01:33.445847 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:33.445782    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:01:33.445847 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:33.445813    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:01:33.491246 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:33.491194    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:34.424735 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:34.424644    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp" event={"ID":"08f613b9-8231-4e10-8a70-5dee789fc9f7","Type":"ContainerStarted","Data":"c1f958f7ae0aa57c8a5553f56d2d8e13cffbca941725d8bfedd50fc9a083fdb1"}
Oct 30 22:01:34.424735 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:34.424709    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp" event={"ID":"08f613b9-8231-4e10-8a70-5dee789fc9f7","Type":"ContainerStarted","Data":"7be2ce43612c0b1d93e1ddb9af197e93a3086138bd85080870e25338b372a98f"}
Oct 30 22:01:34.620772 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:34.620701    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp" podStartSLOduration=4.620676518 podStartE2EDuration="4.620676518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:34.60279351 +0000 UTC m=+1637.274697470" watchObservedRunningTime="2026-10-30 22:01:34.620676518 +0000 UTC m=+1637.292580456"
Oct 30 22:01:34.794595 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:34.794500    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:34.973682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:34.973638    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:34.982038 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:34.982010    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:35.433723 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:35.433691    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:35.434855 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:35.434829    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:01:35.840047 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:35.839903    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:36.084728 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:36.084457    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:36.100390 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:36.100291    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"]
Oct 30 22:01:36.105195 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:36.105160    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bbcf2648-9v5bm"]
Oct 30 22:01:37.058445 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:37.058408    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"]
Oct 30 22:01:37.106887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:37.106853    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b4b07881-jbhhs"]
Oct 30 22:01:37.454671 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:37.454628    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9110ee28-abf0-46dd-a2e7-deec072b5eb6" path="/var/lib/kubelet/pods/9110ee28-abf0-46dd-a2e7-deec072b5eb6/volumes"
Oct 30 22:01:37.455412 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:37.455392    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e54cc187-16f5-4efa-8d02-1e91b0a99d1c" path="/var/lib/kubelet/pods/e54cc187-16f5-4efa-8d02-1e91b0a99d1c/volumes"
Oct 30 22:01:37.868085 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:37.867984    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:38.059335 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:38.059294    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:38.237282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:38.237199    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:40.738917 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:40.738879    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:41.174849 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:41.174805    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:41.400212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:41.400170    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:43.356517 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:43.356470    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:43.670014 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:43.669971    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:46.464637 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:46.464600    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:46.635486 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:46.635434    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:46.811394 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:46.811302    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:48.364360 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:01:48.364314    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:01:52.186787 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:52.186751    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:01:53.050432 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:53.050391    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"]
Oct 30 22:01:53.099351 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:53.099316    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2789fc2f-lpshv"]
Oct 30 22:01:53.450126 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:53.450082    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b74bfc9b-869c-4641-98dc-091ed8bc00bb" path="/var/lib/kubelet/pods/b74bfc9b-869c-4641-98dc-091ed8bc00bb/volumes"
Oct 30 22:01:56.978909 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:01:56.978867    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:02.085191 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:02.085140    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:07.132918 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:07.132874    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:09.917243 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:09.917207    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:10.083656 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:10.083616    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:11.924469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:11.924428    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:15.044937 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:15.044894    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:15.196608 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:15.196545    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:15.236771 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:15.236716    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"]
Oct 30 22:02:15.236972 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:15.236949    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerName="istio-proxy" containerID="cri-o://d916070f35bd032cea5c5b34265d461c42af8cd2b8a5bfc58dbade6e5fa61032" gracePeriod=30
Oct 30 22:02:15.285529 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:15.285490    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"]
Oct 30 22:02:15.285758 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:15.285732    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r" podUID="832ac4dd-f2fc-47b3-82f6-03253f91ff46" containerName="istio-proxy" containerID="cri-o://10bdbd357e3d1fb12925b211cb1ad66f9802070ecc645646207de6798d409289" gracePeriod=30
Oct 30 22:02:15.289789 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:15.289759    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"]
Oct 30 22:02:15.290018 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:15.289989    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj" podUID="27b11c10-67cb-4786-8f9a-70afaa107f77" containerName="istio-proxy" containerID="cri-o://3a1adcbd62e9cf082fad2d6f7dabdfe7cac37ad5cd13b75cdf522d6fd9142d76" gracePeriod=30
Oct 30 22:02:15.360653 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:15.360474    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:16.068760 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:16.068721    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:16.285036 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:16.284925    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:17.140765 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:17.140726    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:17.466945 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:17.466908    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:18.363790 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:02:18.363750    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:02:18.527206 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.527166    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"]
Oct 30 22:02:18.527426 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.527405    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy" containerID="cri-o://d3c71b55f51be831ccd90ab32288ba5bd03e22efb5cd127107bab0c921057496" gracePeriod=30
Oct 30 22:02:18.576646 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.576604    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"]
Oct 30 22:02:18.576892 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.576863    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy" containerID="cri-o://2e9b09c76cae0dfc74829d6a4eaf9929437250dbc16b700bfe92e749d091ea5b" gracePeriod=30
Oct 30 22:02:18.581327 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.581299    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"]
Oct 30 22:02:18.581551 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.581526    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5" podUID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerName="istio-proxy" containerID="cri-o://abe8dcf371ee3779ae62c82677519870e0d0a21a296f068db885f886606abf8c" gracePeriod=30
Oct 30 22:02:18.627873 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.627791    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"]
Oct 30 22:02:18.628020 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.627995    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w" podUID="0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" containerName="istio-proxy" containerID="cri-o://83c1121e575c1f56c2cb1e3c89536fed17a3828d8259b0ed045764d1e3d1b165" gracePeriod=30
Oct 30 22:02:18.805435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.805400    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"]
Oct 30 22:02:18.805676 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.805651    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv" podUID="bbe524d9-e536-4990-ac0b-8ad75c9fd52e" containerName="istio-proxy" containerID="cri-o://667f837adaf4fe3b18d5f6d7ae00dd52971dfaf8de5cf2be9f354f3f7e40eaef" gracePeriod=30
Oct 30 22:02:18.810167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.810137    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"]
Oct 30 22:02:18.816004 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.810452    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n" podUID="23db1e9a-3c13-4c53-9c92-028efebcec23" containerName="istio-proxy" containerID="cri-o://69aba28005014ce806931e1ebbb4cac213bf7b7ac086d9928236bad04c45cbc4" gracePeriod=30
Oct 30 22:02:18.816589 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.816565    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"]
Oct 30 22:02:18.816833 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.816811    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" podUID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" containerName="istio-proxy" containerID="cri-o://ec47de0245618ac8759325c15d25ca595a629c69f9b980867c547d94346b832b" gracePeriod=30
Oct 30 22:02:18.821196 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.821171    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"]
Oct 30 22:02:18.821470 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.821437    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr" podUID="cbade6b6-381e-4034-a3c7-075ce93c8218" containerName="istio-proxy" containerID="cri-o://d12812762a2d73874883ab158b5a51bdb53c24069a5ae1bf3e48beb524a1dcad" gracePeriod=30
Oct 30 22:02:18.978192 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:18.978157    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:19.594324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:19.588695    5586 scope.go:122] "RemoveContainer" containerID="bbb49e79897a75ba7fca41e78ee5bd64caf3be2fc0a092da281e71e1de0b13bb"
Oct 30 22:02:19.618926 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:19.618288    5586 scope.go:122] "RemoveContainer" containerID="f35812722e70d61f5ed5c1de3508c8a08a0d500f7421db2edcb2540d48e747b2"
Oct 30 22:02:19.668330 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:19.667867    5586 scope.go:122] "RemoveContainer" containerID="32de16a507b7fd7c8623931b7784bff9fece99c37563df43ed94520cd53bb5f3"
Oct 30 22:02:20.365893 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.365856    5586 patch_prober.go:29] interesting pod/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:02:20.366064 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.365919    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj" podUID="27b11c10-67cb-4786-8f9a-70afaa107f77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 22:02:20.677046 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.677006    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 22:02:20.682434 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.682400    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 22:02:20.707929 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.707900    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 22:02:20.720818 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.720768    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istiod-ca-cert\") pod \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") "
Oct 30 22:02:20.720818 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.720821    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-credential-socket\") pod \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") "
Oct 30 22:02:20.721081 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.720961    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-data\") pod \"27b11c10-67cb-4786-8f9a-70afaa107f77\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") "
Oct 30 22:02:20.721081 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.720988    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-envoy\") pod \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") "
Oct 30 22:02:20.721081 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721014    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-data\") pod \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") "
Oct 30 22:02:20.721373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721124    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-podinfo\") pod \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") "
Oct 30 22:02:20.721373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721125    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-credential-socket" pod "832ac4dd-f2fc-47b3-82f6-03253f91ff46" (UID: "832ac4dd-f2fc-47b3-82f6-03253f91ff46"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.721373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721291    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27b11c10-67cb-4786-8f9a-70afaa107f77-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27b11c10-67cb-4786-8f9a-70afaa107f77-istiod-ca-cert\") pod \"27b11c10-67cb-4786-8f9a-70afaa107f77\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") "
Oct 30 22:02:20.721373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721282    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-data" pod "27b11c10-67cb-4786-8f9a-70afaa107f77" (UID: "27b11c10-67cb-4786-8f9a-70afaa107f77"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.721373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721337    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istiod-ca-cert" pod "832ac4dd-f2fc-47b3-82f6-03253f91ff46" (UID: "832ac4dd-f2fc-47b3-82f6-03253f91ff46"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:20.721373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721359    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-data" pod "832ac4dd-f2fc-47b3-82f6-03253f91ff46" (UID: "832ac4dd-f2fc-47b3-82f6-03253f91ff46"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.721373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721363    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-envoy\") pod \"27b11c10-67cb-4786-8f9a-70afaa107f77\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") "
Oct 30 22:02:20.721693 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721466    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/832ac4dd-f2fc-47b3-82f6-03253f91ff46-kube-api-access-f9gs7\" (UniqueName: \"kubernetes.io/projected/832ac4dd-f2fc-47b3-82f6-03253f91ff46-kube-api-access-f9gs7\") pod \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") "
Oct 30 22:02:20.721693 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721497    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-workload-certs\") pod \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") "
Oct 30 22:02:20.721693 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721555    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-podinfo\") pod \"27b11c10-67cb-4786-8f9a-70afaa107f77\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") "
Oct 30 22:02:20.721693 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721594    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-workload-certs\") pod \"27b11c10-67cb-4786-8f9a-70afaa107f77\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") "
Oct 30 22:02:20.721693 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721627    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-credential-socket\") pod \"27b11c10-67cb-4786-8f9a-70afaa107f77\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") "
Oct 30 22:02:20.721693 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721680    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27b11c10-67cb-4786-8f9a-70afaa107f77-istiod-ca-cert" pod "27b11c10-67cb-4786-8f9a-70afaa107f77" (UID: "27b11c10-67cb-4786-8f9a-70afaa107f77"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:20.721970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721756    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-workload-socket\") pod \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") "
Oct 30 22:02:20.721970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721799    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-workload-socket\") pod \"27b11c10-67cb-4786-8f9a-70afaa107f77\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") "
Oct 30 22:02:20.721970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721818    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27b11c10-67cb-4786-8f9a-70afaa107f77-kube-api-access-vqmgj\" (UniqueName: \"kubernetes.io/projected/27b11c10-67cb-4786-8f9a-70afaa107f77-kube-api-access-vqmgj\") pod \"27b11c10-67cb-4786-8f9a-70afaa107f77\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") "
Oct 30 22:02:20.721970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721834    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-token\" (UniqueName: \"kubernetes.io/projected/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-token\") pod \"27b11c10-67cb-4786-8f9a-70afaa107f77\" (UID: \"27b11c10-67cb-4786-8f9a-70afaa107f77\") "
Oct 30 22:02:20.721970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721908    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-token\" (UniqueName: \"kubernetes.io/projected/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-token\") pod \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\" (UID: \"832ac4dd-f2fc-47b3-82f6-03253f91ff46\") "
Oct 30 22:02:20.721970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721928    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-workload-certs" pod "832ac4dd-f2fc-47b3-82f6-03253f91ff46" (UID: "832ac4dd-f2fc-47b3-82f6-03253f91ff46"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.721970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.721942    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-workload-certs" pod "27b11c10-67cb-4786-8f9a-70afaa107f77" (UID: "27b11c10-67cb-4786-8f9a-70afaa107f77"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.722286 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.722117    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-workload-socket" pod "27b11c10-67cb-4786-8f9a-70afaa107f77" (UID: "27b11c10-67cb-4786-8f9a-70afaa107f77"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.722752 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.722726    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.722752 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.722754    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.722931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.722770    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.722931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.722786    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.722931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.722819    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.722931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.722833    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.722931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.722846    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.722931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.722859    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27b11c10-67cb-4786-8f9a-70afaa107f77-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.723326 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.723086    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-credential-socket" pod "27b11c10-67cb-4786-8f9a-70afaa107f77" (UID: "27b11c10-67cb-4786-8f9a-70afaa107f77"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.723326 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.723287    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-workload-socket" pod "832ac4dd-f2fc-47b3-82f6-03253f91ff46" (UID: "832ac4dd-f2fc-47b3-82f6-03253f91ff46"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.724592 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.724563    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-envoy" pod "27b11c10-67cb-4786-8f9a-70afaa107f77" (UID: "27b11c10-67cb-4786-8f9a-70afaa107f77"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.724857 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.724794    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-envoy" pod "832ac4dd-f2fc-47b3-82f6-03253f91ff46" (UID: "832ac4dd-f2fc-47b3-82f6-03253f91ff46"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.724857 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.724840    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-token" pod "832ac4dd-f2fc-47b3-82f6-03253f91ff46" (UID: "832ac4dd-f2fc-47b3-82f6-03253f91ff46"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:20.725955 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.725928    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-token" pod "27b11c10-67cb-4786-8f9a-70afaa107f77" (UID: "27b11c10-67cb-4786-8f9a-70afaa107f77"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:20.726410 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.726381    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/832ac4dd-f2fc-47b3-82f6-03253f91ff46-kube-api-access-f9gs7" pod "832ac4dd-f2fc-47b3-82f6-03253f91ff46" (UID: "832ac4dd-f2fc-47b3-82f6-03253f91ff46"). InnerVolumeSpecName "kube-api-access-f9gs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:20.726410 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.726401    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27b11c10-67cb-4786-8f9a-70afaa107f77-kube-api-access-vqmgj" pod "27b11c10-67cb-4786-8f9a-70afaa107f77" (UID: "27b11c10-67cb-4786-8f9a-70afaa107f77"). InnerVolumeSpecName "kube-api-access-vqmgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:20.726678 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.726648    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-podinfo" pod "27b11c10-67cb-4786-8f9a-70afaa107f77" (UID: "27b11c10-67cb-4786-8f9a-70afaa107f77"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:20.727818 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.727754    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-podinfo" pod "832ac4dd-f2fc-47b3-82f6-03253f91ff46" (UID: "832ac4dd-f2fc-47b3-82f6-03253f91ff46"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:20.730210 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.730184    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"]
Oct 30 22:02:20.731000 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.730981    5586 cpu_manager.go:414] "containerMap: removing container" podUID="fc730edc-8621-499c-a05a-22dc848e2727" containerName="istio-proxy"
Oct 30 22:02:20.731086 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731000    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc730edc-8621-499c-a05a-22dc848e2727" containerName="istio-proxy"
Oct 30 22:02:20.731086 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731032    5586 cpu_manager.go:414] "containerMap: removing container" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerName="istio-proxy"
Oct 30 22:02:20.731086 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731039    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerName="istio-proxy"
Oct 30 22:02:20.731086 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731051    5586 cpu_manager.go:414] "containerMap: removing container" podUID="0561cf7c-914f-4660-8b42-ecb3c42bec98" containerName="cleanup"
Oct 30 22:02:20.731086 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731059    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0561cf7c-914f-4660-8b42-ecb3c42bec98" containerName="cleanup"
Oct 30 22:02:20.731086 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731081    5586 cpu_manager.go:414] "containerMap: removing container" podUID="832ac4dd-f2fc-47b3-82f6-03253f91ff46" containerName="istio-proxy"
Oct 30 22:02:20.731399 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731089    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="832ac4dd-f2fc-47b3-82f6-03253f91ff46" containerName="istio-proxy"
Oct 30 22:02:20.731399 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731146    5586 cpu_manager.go:414] "containerMap: removing container" podUID="f2822b90-308a-4149-a838-3e23dccbcb62" containerName="cleanup"
Oct 30 22:02:20.731399 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731154    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2822b90-308a-4149-a838-3e23dccbcb62" containerName="cleanup"
Oct 30 22:02:20.731399 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731182    5586 cpu_manager.go:414] "containerMap: removing container" podUID="27b11c10-67cb-4786-8f9a-70afaa107f77" containerName="istio-proxy"
Oct 30 22:02:20.731399 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731190    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27b11c10-67cb-4786-8f9a-70afaa107f77" containerName="istio-proxy"
Oct 30 22:02:20.731641 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731471    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="26b31903-7a3a-4f41-b955-764d32eafb50" containerName="cleanup"
Oct 30 22:02:20.731641 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731488    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="27b11c10-67cb-4786-8f9a-70afaa107f77" containerName="istio-proxy"
Oct 30 22:02:20.731641 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731504    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2822b90-308a-4149-a838-3e23dccbcb62" containerName="cleanup"
Oct 30 22:02:20.731641 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731515    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerName="istio-proxy"
Oct 30 22:02:20.731641 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731528    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="11ca645b-a522-40ff-bd25-c1ee49f02e58" containerName="cleanup"
Oct 30 22:02:20.731641 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.731538    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="832ac4dd-f2fc-47b3-82f6-03253f91ff46" containerName="istio-proxy"
Oct 30 22:02:20.736769 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.736744    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.806965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.806922    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"]
Oct 30 22:02:20.823777 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.823745    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-99f69c-openshift-default-dockercfg-rhqwf\""
Oct 30 22:02:20.823965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.823919    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-data\") pod \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") "
Oct 30 22:02:20.824027 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.824005    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-kube-api-access-6259p\" (UniqueName: \"kubernetes.io/projected/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-kube-api-access-6259p\") pod \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") "
Oct 30 22:02:20.824278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.824104    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istiod-ca-cert\") pod \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") "
Oct 30 22:02:20.824278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.824242    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-data" pod "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" (UID: "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.824278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.824260    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-credential-socket\") pod \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") "
Oct 30 22:02:20.824831 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.824543    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-credential-socket" pod "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" (UID: "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.824831 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.824578    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istiod-ca-cert" pod "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" (UID: "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:20.824831 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.824653    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-workload-socket\") pod \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") "
Oct 30 22:02:20.824831 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.824709    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-envoy\") pod \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") "
Oct 30 22:02:20.824831 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.824745    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-workload-certs\") pod \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") "
Oct 30 22:02:20.825111 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.824859    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-token\" (UniqueName: \"kubernetes.io/projected/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-token\") pod \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") "
Oct 30 22:02:20.825111 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.824979    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-podinfo\") pod \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\" (UID: \"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0\") "
Oct 30 22:02:20.825312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.825290    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-workload-certs" pod "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" (UID: "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.825380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.825354    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-workload-socket" pod "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" (UID: "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.825685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.825660    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-token\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.825810 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.825792    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-workload-certs\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.825950 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.825932    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mm6z\" (UniqueName: \"kubernetes.io/projected/a76b490b-c349-4a5c-ba7e-6445596e7ee0-kube-api-access-6mm6z\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.826027 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.826011    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-envoy\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.826158 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.826142    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-podinfo\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.826225 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.826181    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istiod-ca-cert\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.826552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.826530    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-credential-socket\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.826616 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.826604    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-data\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.826797 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.826778    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-workload-socket\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.827030 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827008    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-kube-api-access-6259p" pod "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" (UID: "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0"). InnerVolumeSpecName "kube-api-access-6259p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:20.827126 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827033    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827126 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827055    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827126 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827071    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827126 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827088    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827126 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827105    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqmgj\" (UniqueName: \"kubernetes.io/projected/27b11c10-67cb-4786-8f9a-70afaa107f77-kube-api-access-vqmgj\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827126 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827122    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827138    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827153    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827167    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827181    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827195    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827209    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/832ac4dd-f2fc-47b3-82f6-03253f91ff46-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827223    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27b11c10-67cb-4786-8f9a-70afaa107f77-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827253    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827266    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9gs7\" (UniqueName: \"kubernetes.io/projected/832ac4dd-f2fc-47b3-82f6-03253f91ff46-kube-api-access-f9gs7\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.827973 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.827792    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-podinfo" pod "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" (UID: "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:20.828856 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.828832    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-envoy" pod "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" (UID: "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:20.829317 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.829298    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-token" pod "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" (UID: "1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:20.845445 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.845399    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:20.928808 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.928759    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-podinfo\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.929020 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.928814    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istiod-ca-cert\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.929020 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.928908    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-credential-socket\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.929020 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.928938    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-data\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.929020 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.929014    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-workload-socket\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.929280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.929076    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-token\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.929280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.929128    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-workload-certs\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.929280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.929250    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mm6z\" (UniqueName: \"kubernetes.io/projected/a76b490b-c349-4a5c-ba7e-6445596e7ee0-kube-api-access-6mm6z\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.929440 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.929315    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-envoy\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.929440 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.929389    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-credential-socket\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.929440 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.929403    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.929440 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.929419    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.929440 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.929434    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.929612 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.929450    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6259p\" (UniqueName: \"kubernetes.io/projected/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0-kube-api-access-6259p\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:20.929651 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.929610    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istiod-ca-cert\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.930280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.930218    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-workload-socket\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.930403 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.930285    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-data\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.930585 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.930560    5586 generic.go:356] "Generic (PLEG): container finished" podID="27b11c10-67cb-4786-8f9a-70afaa107f77" containerID="3a1adcbd62e9cf082fad2d6f7dabdfe7cac37ad5cd13b75cdf522d6fd9142d76" exitCode=0
Oct 30 22:02:20.930649 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.930623    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"
Oct 30 22:02:20.930714 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.930645    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj" event={"ID":"27b11c10-67cb-4786-8f9a-70afaa107f77","Type":"ContainerDied","Data":"3a1adcbd62e9cf082fad2d6f7dabdfe7cac37ad5cd13b75cdf522d6fd9142d76"}
Oct 30 22:02:20.930771 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.930713    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj" event={"ID":"27b11c10-67cb-4786-8f9a-70afaa107f77","Type":"ContainerDied","Data":"9493319118c19dd345941c34ebb54269ea974c32a1257d4b8336991416c97cb8"}
Oct 30 22:02:20.930771 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.930745    5586 scope.go:122] "RemoveContainer" containerID="3a1adcbd62e9cf082fad2d6f7dabdfe7cac37ad5cd13b75cdf522d6fd9142d76"
Oct 30 22:02:20.930861 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.930846    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-workload-certs\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.931935 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.931907    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-podinfo\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.932566 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.932542    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-envoy\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.933023 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.932997    5586 generic.go:356] "Generic (PLEG): container finished" podID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" containerID="d916070f35bd032cea5c5b34265d461c42af8cd2b8a5bfc58dbade6e5fa61032" exitCode=0
Oct 30 22:02:20.933187 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.933168    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" event={"ID":"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0","Type":"ContainerDied","Data":"d916070f35bd032cea5c5b34265d461c42af8cd2b8a5bfc58dbade6e5fa61032"}
Oct 30 22:02:20.933280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.933226    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd" event={"ID":"1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0","Type":"ContainerDied","Data":"f1c61c0ff1793767eba2aadc91bf3665f8197b711b4cd7fb202d1b8661f1eab4"}
Oct 30 22:02:20.933353 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.933338    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"
Oct 30 22:02:20.940275 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.940203    5586 generic.go:356] "Generic (PLEG): container finished" podID="832ac4dd-f2fc-47b3-82f6-03253f91ff46" containerID="10bdbd357e3d1fb12925b211cb1ad66f9802070ecc645646207de6798d409289" exitCode=0
Oct 30 22:02:20.940365 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.940333    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"
Oct 30 22:02:20.940553 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.940531    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r" event={"ID":"832ac4dd-f2fc-47b3-82f6-03253f91ff46","Type":"ContainerDied","Data":"10bdbd357e3d1fb12925b211cb1ad66f9802070ecc645646207de6798d409289"}
Oct 30 22:02:20.940619 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.940592    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r" event={"ID":"832ac4dd-f2fc-47b3-82f6-03253f91ff46","Type":"ContainerDied","Data":"e6c12fbfd75b00e18e66a917d5bf9b8753ef9253c280c6808253a3703157b04a"}
Oct 30 22:02:20.951677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.951652    5586 scope.go:122] "RemoveContainer" containerID="3a1adcbd62e9cf082fad2d6f7dabdfe7cac37ad5cd13b75cdf522d6fd9142d76"
Oct 30 22:02:20.952057 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:02:20.952021    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a1adcbd62e9cf082fad2d6f7dabdfe7cac37ad5cd13b75cdf522d6fd9142d76\": container with ID starting with 3a1adcbd62e9cf082fad2d6f7dabdfe7cac37ad5cd13b75cdf522d6fd9142d76 not found: ID does not exist" containerID="3a1adcbd62e9cf082fad2d6f7dabdfe7cac37ad5cd13b75cdf522d6fd9142d76"
Oct 30 22:02:20.952318 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.952067    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a1adcbd62e9cf082fad2d6f7dabdfe7cac37ad5cd13b75cdf522d6fd9142d76"} err="failed to get container status \"3a1adcbd62e9cf082fad2d6f7dabdfe7cac37ad5cd13b75cdf522d6fd9142d76\": rpc error: code = NotFound desc = could not find container \"3a1adcbd62e9cf082fad2d6f7dabdfe7cac37ad5cd13b75cdf522d6fd9142d76\": container with ID starting with 3a1adcbd62e9cf082fad2d6f7dabdfe7cac37ad5cd13b75cdf522d6fd9142d76 not found: ID does not exist"
Oct 30 22:02:20.952318 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.952090    5586 scope.go:122] "RemoveContainer" containerID="d916070f35bd032cea5c5b34265d461c42af8cd2b8a5bfc58dbade6e5fa61032"
Oct 30 22:02:20.964352 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.964320    5586 scope.go:122] "RemoveContainer" containerID="d916070f35bd032cea5c5b34265d461c42af8cd2b8a5bfc58dbade6e5fa61032"
Oct 30 22:02:20.965214 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:02:20.965190    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d916070f35bd032cea5c5b34265d461c42af8cd2b8a5bfc58dbade6e5fa61032\": container with ID starting with d916070f35bd032cea5c5b34265d461c42af8cd2b8a5bfc58dbade6e5fa61032 not found: ID does not exist" containerID="d916070f35bd032cea5c5b34265d461c42af8cd2b8a5bfc58dbade6e5fa61032"
Oct 30 22:02:20.965368 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.965223    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d916070f35bd032cea5c5b34265d461c42af8cd2b8a5bfc58dbade6e5fa61032"} err="failed to get container status \"d916070f35bd032cea5c5b34265d461c42af8cd2b8a5bfc58dbade6e5fa61032\": rpc error: code = NotFound desc = could not find container \"d916070f35bd032cea5c5b34265d461c42af8cd2b8a5bfc58dbade6e5fa61032\": container with ID starting with d916070f35bd032cea5c5b34265d461c42af8cd2b8a5bfc58dbade6e5fa61032 not found: ID does not exist"
Oct 30 22:02:20.965368 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.965268    5586 scope.go:122] "RemoveContainer" containerID="10bdbd357e3d1fb12925b211cb1ad66f9802070ecc645646207de6798d409289"
Oct 30 22:02:20.976446 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.976425    5586 scope.go:122] "RemoveContainer" containerID="10bdbd357e3d1fb12925b211cb1ad66f9802070ecc645646207de6798d409289"
Oct 30 22:02:20.976804 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:02:20.976776    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10bdbd357e3d1fb12925b211cb1ad66f9802070ecc645646207de6798d409289\": container with ID starting with 10bdbd357e3d1fb12925b211cb1ad66f9802070ecc645646207de6798d409289 not found: ID does not exist" containerID="10bdbd357e3d1fb12925b211cb1ad66f9802070ecc645646207de6798d409289"
Oct 30 22:02:20.976894 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.976815    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10bdbd357e3d1fb12925b211cb1ad66f9802070ecc645646207de6798d409289"} err="failed to get container status \"10bdbd357e3d1fb12925b211cb1ad66f9802070ecc645646207de6798d409289\": rpc error: code = NotFound desc = could not find container \"10bdbd357e3d1fb12925b211cb1ad66f9802070ecc645646207de6798d409289\": container with ID starting with 10bdbd357e3d1fb12925b211cb1ad66f9802070ecc645646207de6798d409289 not found: ID does not exist"
Oct 30 22:02:20.987907 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.987867    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mm6z\" (UniqueName: \"kubernetes.io/projected/a76b490b-c349-4a5c-ba7e-6445596e7ee0-kube-api-access-6mm6z\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:20.992107 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:20.992073    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-token\") pod \"e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") " pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:21.027005 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.026964    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:21.057589 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.057557    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:21.358773 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.358709    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"]
Oct 30 22:02:21.362973 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.362944    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-rjmwd"]
Oct 30 22:02:21.450333 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.450221    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0" path="/var/lib/kubelet/pods/1bf7cdd7-fc8d-4d79-a3e7-cc24f885ccd0/volumes"
Oct 30 22:02:21.474535 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.474502    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"]
Oct 30 22:02:21.480669 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.480638    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-4275r"]
Oct 30 22:02:21.571587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.571548    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"]
Oct 30 22:02:21.584805 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.584751    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:02:21.584938 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.584848    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:02:21.584938 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.584888    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:02:21.588033 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.588003    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"]
Oct 30 22:02:21.592486 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.592457    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-h7rdj"]
Oct 30 22:02:21.614731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.614690    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:02:21.614927 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.614798    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv" podUID="bbe524d9-e536-4990-ac0b-8ad75c9fd52e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:02:21.954538 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.954492    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm" event={"ID":"a76b490b-c349-4a5c-ba7e-6445596e7ee0","Type":"ContainerStarted","Data":"e2373d3d6e42c85d745de5944cf1c7f7290636749099a83030a7a2c142bc01db"}
Oct 30 22:02:21.955015 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:21.954553    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm" event={"ID":"a76b490b-c349-4a5c-ba7e-6445596e7ee0","Type":"ContainerStarted","Data":"4d62ce38a71e86f333494ae09c284bd29908ee5e674ef7113f8e79b13799916d"}
Oct 30 22:02:22.058655 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:22.058621    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:22.064721 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:22.064689    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:22.074344 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:22.074281    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm" podStartSLOduration=2.07426099 podStartE2EDuration="2.07426099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:22.069446185 +0000 UTC m=+1684.741350149" watchObservedRunningTime="2026-10-30 22:02:22.07426099 +0000 UTC m=+1684.746164925"
Oct 30 22:02:22.372386 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:22.372293    5586 patch_prober.go:29] interesting pod/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:02:22.372386 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:22.372353    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w" podUID="0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:02:22.721460 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:22.721415    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:22.807262 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:22.807207    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:02:22.807442 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:22.807292    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n" podUID="23db1e9a-3c13-4c53-9c92-028efebcec23" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:02:22.962477 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:22.962448    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:22.963516 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:22.963495    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:02:23.449326 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:23.449275    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27b11c10-67cb-4786-8f9a-70afaa107f77" path="/var/lib/kubelet/pods/27b11c10-67cb-4786-8f9a-70afaa107f77/volumes"
Oct 30 22:02:23.449959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:23.449942    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="832ac4dd-f2fc-47b3-82f6-03253f91ff46" path="/var/lib/kubelet/pods/832ac4dd-f2fc-47b3-82f6-03253f91ff46/volumes"
Oct 30 22:02:23.570954 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:23.570921    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:02:23.571076 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:23.570979    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr" podUID="cbade6b6-381e-4034-a3c7-075ce93c8218" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:02:24.012999 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.012799    5586 generic.go:356] "Generic (PLEG): container finished" podID="0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" containerID="83c1121e575c1f56c2cb1e3c89536fed17a3828d8259b0ed045764d1e3d1b165" exitCode=0
Oct 30 22:02:24.012999 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.012930    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w" event={"ID":"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a","Type":"ContainerDied","Data":"83c1121e575c1f56c2cb1e3c89536fed17a3828d8259b0ed045764d1e3d1b165"}
Oct 30 22:02:24.018124 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.018022    5586 generic.go:356] "Generic (PLEG): container finished" podID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerID="abe8dcf371ee3779ae62c82677519870e0d0a21a296f068db885f886606abf8c" exitCode=0
Oct 30 22:02:24.018487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.018333    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5" event={"ID":"93475d2f-6f42-47e4-9c1e-44cd9080924c","Type":"ContainerDied","Data":"abe8dcf371ee3779ae62c82677519870e0d0a21a296f068db885f886606abf8c"}
Oct 30 22:02:24.018487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.018397    5586 scope.go:122] "RemoveContainer" containerID="13a09357e9e29b57a29134c2c276d66a096fddf2e9d2aa3d197a7cd71499f430"
Oct 30 22:02:24.026938 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.026749    5586 generic.go:356] "Generic (PLEG): container finished" podID="c3360def-3e78-4997-98af-fefd89f483ee" containerID="2e9b09c76cae0dfc74829d6a4eaf9929437250dbc16b700bfe92e749d091ea5b" exitCode=0
Oct 30 22:02:24.026938 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.026863    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" event={"ID":"c3360def-3e78-4997-98af-fefd89f483ee","Type":"ContainerDied","Data":"2e9b09c76cae0dfc74829d6a4eaf9929437250dbc16b700bfe92e749d091ea5b"}
Oct 30 22:02:24.029247 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.029018    5586 generic.go:356] "Generic (PLEG): container finished" podID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerID="d3c71b55f51be831ccd90ab32288ba5bd03e22efb5cd127107bab0c921057496" exitCode=0
Oct 30 22:02:24.029247 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.029114    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" event={"ID":"ee1cab02-9630-4c82-b5ed-f3f00e4882ed","Type":"ContainerDied","Data":"d3c71b55f51be831ccd90ab32288ba5bd03e22efb5cd127107bab0c921057496"}
Oct 30 22:02:24.029247 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.029164    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl" event={"ID":"ee1cab02-9630-4c82-b5ed-f3f00e4882ed","Type":"ContainerDied","Data":"799b8b8a3cd2158d73bfdc48e78613a5a000e7c32eed9e727841374914780f43"}
Oct 30 22:02:24.029247 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.029192    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="799b8b8a3cd2158d73bfdc48e78613a5a000e7c32eed9e727841374914780f43"
Oct 30 22:02:24.037790 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.037615    5586 generic.go:356] "Generic (PLEG): container finished" podID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" containerID="ec47de0245618ac8759325c15d25ca595a629c69f9b980867c547d94346b832b" exitCode=0
Oct 30 22:02:24.037790 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.037723    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" event={"ID":"d88dfc7e-36a5-4dd1-9814-2b966c89b085","Type":"ContainerDied","Data":"ec47de0245618ac8759325c15d25ca595a629c69f9b980867c547d94346b832b"}
Oct 30 22:02:24.039734 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.039615    5586 generic.go:356] "Generic (PLEG): container finished" podID="bbe524d9-e536-4990-ac0b-8ad75c9fd52e" containerID="667f837adaf4fe3b18d5f6d7ae00dd52971dfaf8de5cf2be9f354f3f7e40eaef" exitCode=0
Oct 30 22:02:24.039734 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.039684    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv" event={"ID":"bbe524d9-e536-4990-ac0b-8ad75c9fd52e","Type":"ContainerDied","Data":"667f837adaf4fe3b18d5f6d7ae00dd52971dfaf8de5cf2be9f354f3f7e40eaef"}
Oct 30 22:02:24.041781 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.041755    5586 generic.go:356] "Generic (PLEG): container finished" podID="23db1e9a-3c13-4c53-9c92-028efebcec23" containerID="69aba28005014ce806931e1ebbb4cac213bf7b7ac086d9928236bad04c45cbc4" exitCode=0
Oct 30 22:02:24.041959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.041939    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n" event={"ID":"23db1e9a-3c13-4c53-9c92-028efebcec23","Type":"ContainerDied","Data":"69aba28005014ce806931e1ebbb4cac213bf7b7ac086d9928236bad04c45cbc4"}
Oct 30 22:02:24.043803 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.043777    5586 generic.go:356] "Generic (PLEG): container finished" podID="cbade6b6-381e-4034-a3c7-075ce93c8218" containerID="d12812762a2d73874883ab158b5a51bdb53c24069a5ae1bf3e48beb524a1dcad" exitCode=0
Oct 30 22:02:24.044361 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.044311    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr" event={"ID":"cbade6b6-381e-4034-a3c7-075ce93c8218","Type":"ContainerDied","Data":"d12812762a2d73874883ab158b5a51bdb53c24069a5ae1bf3e48beb524a1dcad"}
Oct 30 22:02:24.133931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.132580    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 22:02:24.141863 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.141786    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 22:02:24.156657 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.156424    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 22:02:24.158903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.158875    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 22:02:24.206560 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.206517    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-credential-socket\") pod \"c3360def-3e78-4997-98af-fefd89f483ee\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") "
Oct 30 22:02:24.206809 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.206571    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-credential-socket\") pod \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") "
Oct 30 22:02:24.206809 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.206612    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3360def-3e78-4997-98af-fefd89f483ee-kube-api-access-xstvs\" (UniqueName: \"kubernetes.io/projected/c3360def-3e78-4997-98af-fefd89f483ee-kube-api-access-xstvs\") pod \"c3360def-3e78-4997-98af-fefd89f483ee\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") "
Oct 30 22:02:24.206809 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.206691    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3360def-3e78-4997-98af-fefd89f483ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3360def-3e78-4997-98af-fefd89f483ee-istiod-ca-cert\") pod \"c3360def-3e78-4997-98af-fefd89f483ee\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") "
Oct 30 22:02:24.206809 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.206727    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-podinfo\") pod \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") "
Oct 30 22:02:24.206809 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.206783    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-workload-socket\") pod \"c3360def-3e78-4997-98af-fefd89f483ee\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") "
Oct 30 22:02:24.207106 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.206864    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-credential-socket\") pod \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") "
Oct 30 22:02:24.207106 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.206918    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-workload-certs\") pod \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") "
Oct 30 22:02:24.207106 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.206925    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-credential-socket" pod "c3360def-3e78-4997-98af-fefd89f483ee" (UID: "c3360def-3e78-4997-98af-fefd89f483ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.207106 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.206949    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-envoy\") pod \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") "
Oct 30 22:02:24.207106 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207005    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-envoy\") pod \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") "
Oct 30 22:02:24.207106 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207049    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-token\" (UniqueName: \"kubernetes.io/projected/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-token\") pod \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") "
Oct 30 22:02:24.207106 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207078    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c3360def-3e78-4997-98af-fefd89f483ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3360def-3e78-4997-98af-fefd89f483ee-istio-podinfo\") pod \"c3360def-3e78-4997-98af-fefd89f483ee\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") "
Oct 30 22:02:24.207502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207141    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-workload-certs\") pod \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") "
Oct 30 22:02:24.207502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207282    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-istio-envoy\") pod \"c3360def-3e78-4997-98af-fefd89f483ee\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") "
Oct 30 22:02:24.207502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207334    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-podinfo\") pod \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") "
Oct 30 22:02:24.207502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207368    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-workload-socket\") pod \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") "
Oct 30 22:02:24.207502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207395    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-workload-socket\") pod \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") "
Oct 30 22:02:24.207502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207474    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-istio-data\") pod \"c3360def-3e78-4997-98af-fefd89f483ee\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") "
Oct 30 22:02:24.207794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207528    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istiod-ca-cert\") pod \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") "
Oct 30 22:02:24.207794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207557    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-data\") pod \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") "
Oct 30 22:02:24.207794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207588    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-kube-api-access-5dmbw\" (UniqueName: \"kubernetes.io/projected/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-kube-api-access-5dmbw\") pod \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") "
Oct 30 22:02:24.207794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207636    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-kube-api-access-d9drv\" (UniqueName: \"kubernetes.io/projected/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-kube-api-access-d9drv\") pod \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") "
Oct 30 22:02:24.207794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207729    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3360def-3e78-4997-98af-fefd89f483ee-istio-token\" (UniqueName: \"kubernetes.io/projected/c3360def-3e78-4997-98af-fefd89f483ee-istio-token\") pod \"c3360def-3e78-4997-98af-fefd89f483ee\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") "
Oct 30 22:02:24.208044 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207806    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-workload-certs\") pod \"c3360def-3e78-4997-98af-fefd89f483ee\" (UID: \"c3360def-3e78-4997-98af-fefd89f483ee\") "
Oct 30 22:02:24.208044 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207834    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istiod-ca-cert\") pod \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") "
Oct 30 22:02:24.208044 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207903    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-token\" (UniqueName: \"kubernetes.io/projected/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-token\") pod \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\" (UID: \"ee1cab02-9630-4c82-b5ed-f3f00e4882ed\") "
Oct 30 22:02:24.208044 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207922    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-credential-socket" pod "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" (UID: "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.208044 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.207932    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-data\") pod \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\" (UID: \"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a\") "
Oct 30 22:02:24.208322 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.208145    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-data" pod "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" (UID: "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.209398 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.208605    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-workload-certs" pod "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" (UID: "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.209398 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.208910    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-workload-socket" pod "c3360def-3e78-4997-98af-fefd89f483ee" (UID: "c3360def-3e78-4997-98af-fefd89f483ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.209398 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.209118    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-istio-data" pod "c3360def-3e78-4997-98af-fefd89f483ee" (UID: "c3360def-3e78-4997-98af-fefd89f483ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.209398 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.209274    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.209398 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.209297    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.209398 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.209329    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.209398 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.209342    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.209398 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.209358    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.209870 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.209417    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-credential-socket" pod "ee1cab02-9630-4c82-b5ed-f3f00e4882ed" (UID: "ee1cab02-9630-4c82-b5ed-f3f00e4882ed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.209870 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.209737    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3360def-3e78-4997-98af-fefd89f483ee-istiod-ca-cert" pod "c3360def-3e78-4997-98af-fefd89f483ee" (UID: "c3360def-3e78-4997-98af-fefd89f483ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:24.210362 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.210106    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istiod-ca-cert" pod "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" (UID: "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:24.212104 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.211429    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-data" pod "ee1cab02-9630-4c82-b5ed-f3f00e4882ed" (UID: "ee1cab02-9630-4c82-b5ed-f3f00e4882ed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.212507 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.212470    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-workload-certs" pod "c3360def-3e78-4997-98af-fefd89f483ee" (UID: "c3360def-3e78-4997-98af-fefd89f483ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.213048 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.212664    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istiod-ca-cert" pod "ee1cab02-9630-4c82-b5ed-f3f00e4882ed" (UID: "ee1cab02-9630-4c82-b5ed-f3f00e4882ed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:24.213048 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.212879    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-workload-certs" pod "ee1cab02-9630-4c82-b5ed-f3f00e4882ed" (UID: "ee1cab02-9630-4c82-b5ed-f3f00e4882ed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.213638 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.213424    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-workload-socket" pod "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" (UID: "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.213749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.213652    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-workload-socket" pod "ee1cab02-9630-4c82-b5ed-f3f00e4882ed" (UID: "ee1cab02-9630-4c82-b5ed-f3f00e4882ed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.217733 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.217702    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-istio-envoy" pod "c3360def-3e78-4997-98af-fefd89f483ee" (UID: "c3360def-3e78-4997-98af-fefd89f483ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.217833 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.217769    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3360def-3e78-4997-98af-fefd89f483ee-istio-token" pod "c3360def-3e78-4997-98af-fefd89f483ee" (UID: "c3360def-3e78-4997-98af-fefd89f483ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.217833 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.217803    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-kube-api-access-5dmbw" pod "ee1cab02-9630-4c82-b5ed-f3f00e4882ed" (UID: "ee1cab02-9630-4c82-b5ed-f3f00e4882ed"). InnerVolumeSpecName "kube-api-access-5dmbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.218108 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.217831    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-token" pod "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" (UID: "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.218108 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.217934    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-kube-api-access-d9drv" pod "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" (UID: "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a"). InnerVolumeSpecName "kube-api-access-d9drv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.218108 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.217992    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-envoy" pod "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" (UID: "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.218108 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.218013    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-envoy" pod "ee1cab02-9630-4c82-b5ed-f3f00e4882ed" (UID: "ee1cab02-9630-4c82-b5ed-f3f00e4882ed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.220724 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.220432    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-podinfo" pod "ee1cab02-9630-4c82-b5ed-f3f00e4882ed" (UID: "ee1cab02-9630-4c82-b5ed-f3f00e4882ed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:24.220724 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.220456    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-podinfo" pod "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" (UID: "0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:24.220724 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.220463    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3360def-3e78-4997-98af-fefd89f483ee-istio-podinfo" pod "c3360def-3e78-4997-98af-fefd89f483ee" (UID: "c3360def-3e78-4997-98af-fefd89f483ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:24.223162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.223139    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3360def-3e78-4997-98af-fefd89f483ee-kube-api-access-xstvs" pod "c3360def-3e78-4997-98af-fefd89f483ee" (UID: "c3360def-3e78-4997-98af-fefd89f483ee"). InnerVolumeSpecName "kube-api-access-xstvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.227125 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.227091    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 22:02:24.229068 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.229029    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-token" pod "ee1cab02-9630-4c82-b5ed-f3f00e4882ed" (UID: "ee1cab02-9630-4c82-b5ed-f3f00e4882ed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.311677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.311532    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-token\" (UniqueName: \"kubernetes.io/projected/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-token\") pod \"93475d2f-6f42-47e4-9c1e-44cd9080924c\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") "
Oct 30 22:02:24.312311 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.312204    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-workload-certs\") pod \"93475d2f-6f42-47e4-9c1e-44cd9080924c\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") "
Oct 30 22:02:24.313376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.312441    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-credential-socket\") pod \"23db1e9a-3c13-4c53-9c92-028efebcec23\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") "
Oct 30 22:02:24.313376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.312517    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-podinfo\") pod \"93475d2f-6f42-47e4-9c1e-44cd9080924c\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") "
Oct 30 22:02:24.313376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.312583    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-envoy\") pod \"23db1e9a-3c13-4c53-9c92-028efebcec23\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") "
Oct 30 22:02:24.313376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.312615    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-token\" (UniqueName: \"kubernetes.io/projected/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-token\") pod \"23db1e9a-3c13-4c53-9c92-028efebcec23\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") "
Oct 30 22:02:24.313376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.312658    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23db1e9a-3c13-4c53-9c92-028efebcec23-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23db1e9a-3c13-4c53-9c92-028efebcec23-istiod-ca-cert\") pod \"23db1e9a-3c13-4c53-9c92-028efebcec23\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") "
Oct 30 22:02:24.313376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.312703    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-workload-socket\") pod \"23db1e9a-3c13-4c53-9c92-028efebcec23\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") "
Oct 30 22:02:24.313376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.312798    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-workload-certs\") pod \"23db1e9a-3c13-4c53-9c92-028efebcec23\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") "
Oct 30 22:02:24.313376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.312880    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-podinfo\") pod \"23db1e9a-3c13-4c53-9c92-028efebcec23\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") "
Oct 30 22:02:24.313376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.312908    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93475d2f-6f42-47e4-9c1e-44cd9080924c-kube-api-access-hmkkx\" (UniqueName: \"kubernetes.io/projected/93475d2f-6f42-47e4-9c1e-44cd9080924c-kube-api-access-hmkkx\") pod \"93475d2f-6f42-47e4-9c1e-44cd9080924c\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") "
Oct 30 22:02:24.313376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.312942    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-workload-socket\") pod \"93475d2f-6f42-47e4-9c1e-44cd9080924c\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") "
Oct 30 22:02:24.313376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.312994    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-credential-socket\") pod \"93475d2f-6f42-47e4-9c1e-44cd9080924c\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") "
Oct 30 22:02:24.313376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.313033    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93475d2f-6f42-47e4-9c1e-44cd9080924c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93475d2f-6f42-47e4-9c1e-44cd9080924c-istiod-ca-cert\") pod \"93475d2f-6f42-47e4-9c1e-44cd9080924c\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") "
Oct 30 22:02:24.313376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.313076    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-data\") pod \"93475d2f-6f42-47e4-9c1e-44cd9080924c\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") "
Oct 30 22:02:24.313376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.313116    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-data\") pod \"23db1e9a-3c13-4c53-9c92-028efebcec23\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") "
Oct 30 22:02:24.314082 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.313881    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-workload-certs" pod "93475d2f-6f42-47e4-9c1e-44cd9080924c" (UID: "93475d2f-6f42-47e4-9c1e-44cd9080924c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.315303 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.314283    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-workload-socket" pod "93475d2f-6f42-47e4-9c1e-44cd9080924c" (UID: "93475d2f-6f42-47e4-9c1e-44cd9080924c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.315303 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.314326    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23db1e9a-3c13-4c53-9c92-028efebcec23-kube-api-access-rksnh\" (UniqueName: \"kubernetes.io/projected/23db1e9a-3c13-4c53-9c92-028efebcec23-kube-api-access-rksnh\") pod \"23db1e9a-3c13-4c53-9c92-028efebcec23\" (UID: \"23db1e9a-3c13-4c53-9c92-028efebcec23\") "
Oct 30 22:02:24.315303 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.314371    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-envoy\") pod \"93475d2f-6f42-47e4-9c1e-44cd9080924c\" (UID: \"93475d2f-6f42-47e4-9c1e-44cd9080924c\") "
Oct 30 22:02:24.315303 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.315225    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-credential-socket" pod "23db1e9a-3c13-4c53-9c92-028efebcec23" (UID: "23db1e9a-3c13-4c53-9c92-028efebcec23"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.316303 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.315940    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3360def-3e78-4997-98af-fefd89f483ee-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.316303 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.315973    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.316303 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.316009    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.316515 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.316357    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-workload-socket" pod "23db1e9a-3c13-4c53-9c92-028efebcec23" (UID: "23db1e9a-3c13-4c53-9c92-028efebcec23"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.316775    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-podinfo" pod "23db1e9a-3c13-4c53-9c92-028efebcec23" (UID: "23db1e9a-3c13-4c53-9c92-028efebcec23"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.317012    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-token" pod "93475d2f-6f42-47e4-9c1e-44cd9080924c" (UID: "93475d2f-6f42-47e4-9c1e-44cd9080924c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.317030    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-workload-certs" pod "23db1e9a-3c13-4c53-9c92-028efebcec23" (UID: "23db1e9a-3c13-4c53-9c92-028efebcec23"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.317267    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-data" pod "93475d2f-6f42-47e4-9c1e-44cd9080924c" (UID: "93475d2f-6f42-47e4-9c1e-44cd9080924c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.317458    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-credential-socket" pod "93475d2f-6f42-47e4-9c1e-44cd9080924c" (UID: "93475d2f-6f42-47e4-9c1e-44cd9080924c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.317749    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93475d2f-6f42-47e4-9c1e-44cd9080924c-istiod-ca-cert" pod "93475d2f-6f42-47e4-9c1e-44cd9080924c" (UID: "93475d2f-6f42-47e4-9c1e-44cd9080924c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.316023    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318091    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318108    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318126    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318141    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xstvs\" (UniqueName: \"kubernetes.io/projected/c3360def-3e78-4997-98af-fefd89f483ee-kube-api-access-xstvs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318156    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3360def-3e78-4997-98af-fefd89f483ee-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318170    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318183    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318200    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318214    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318244    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318259    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3360def-3e78-4997-98af-fefd89f483ee-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318272    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318285    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318299    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318315    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318330    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318346    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3360def-3e78-4997-98af-fefd89f483ee-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318360    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318375    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318390    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dmbw\" (UniqueName: \"kubernetes.io/projected/ee1cab02-9630-4c82-b5ed-f3f00e4882ed-kube-api-access-5dmbw\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318405    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9drv\" (UniqueName: \"kubernetes.io/projected/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a-kube-api-access-d9drv\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318434    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-data" pod "23db1e9a-3c13-4c53-9c92-028efebcec23" (UID: "23db1e9a-3c13-4c53-9c92-028efebcec23"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.321281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.318764    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23db1e9a-3c13-4c53-9c92-028efebcec23-istiod-ca-cert" pod "23db1e9a-3c13-4c53-9c92-028efebcec23" (UID: "23db1e9a-3c13-4c53-9c92-028efebcec23"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:24.332835 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.332032    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-envoy" pod "23db1e9a-3c13-4c53-9c92-028efebcec23" (UID: "23db1e9a-3c13-4c53-9c92-028efebcec23"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.332835 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.332035    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23db1e9a-3c13-4c53-9c92-028efebcec23-kube-api-access-rksnh" pod "23db1e9a-3c13-4c53-9c92-028efebcec23" (UID: "23db1e9a-3c13-4c53-9c92-028efebcec23"). InnerVolumeSpecName "kube-api-access-rksnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.332835 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.332190    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-podinfo" pod "93475d2f-6f42-47e4-9c1e-44cd9080924c" (UID: "93475d2f-6f42-47e4-9c1e-44cd9080924c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:24.332835 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.332227    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93475d2f-6f42-47e4-9c1e-44cd9080924c-kube-api-access-hmkkx" pod "93475d2f-6f42-47e4-9c1e-44cd9080924c" (UID: "93475d2f-6f42-47e4-9c1e-44cd9080924c"). InnerVolumeSpecName "kube-api-access-hmkkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.332835 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.332473    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-envoy" pod "93475d2f-6f42-47e4-9c1e-44cd9080924c" (UID: "93475d2f-6f42-47e4-9c1e-44cd9080924c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.334104 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.334078    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-token" pod "23db1e9a-3c13-4c53-9c92-028efebcec23" (UID: "23db1e9a-3c13-4c53-9c92-028efebcec23"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.334652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.334631    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 22:02:24.339570 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.339541    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 22:02:24.381415 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.381379    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 22:02:24.420392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.420354    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-data\") pod \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") "
Oct 30 22:02:24.420629 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.420447    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-workload-certs\") pod \"cbade6b6-381e-4034-a3c7-075ce93c8218\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") "
Oct 30 22:02:24.420629 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.420485    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istiod-ca-cert\") pod \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") "
Oct 30 22:02:24.420629 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.420527    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-kube-api-access-w46dq\" (UniqueName: \"kubernetes.io/projected/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-kube-api-access-w46dq\") pod \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") "
Oct 30 22:02:24.420629 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.420617    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-podinfo\") pod \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") "
Oct 30 22:02:24.420827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.420646    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-workload-socket\") pod \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") "
Oct 30 22:02:24.420827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.420670    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-workload-socket\") pod \"cbade6b6-381e-4034-a3c7-075ce93c8218\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") "
Oct 30 22:02:24.420827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.420740    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-credential-socket\") pod \"cbade6b6-381e-4034-a3c7-075ce93c8218\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") "
Oct 30 22:02:24.420827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.420784    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-workload-certs\") pod \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") "
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.420828    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-data" pod "bbe524d9-e536-4990-ac0b-8ad75c9fd52e" (UID: "bbe524d9-e536-4990-ac0b-8ad75c9fd52e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.420848    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-token\" (UniqueName: \"kubernetes.io/projected/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-token\") pod \"cbade6b6-381e-4034-a3c7-075ce93c8218\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") "
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.420870    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-envoy\") pod \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") "
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.420920    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-envoy\") pod \"cbade6b6-381e-4034-a3c7-075ce93c8218\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") "
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.420973    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-podinfo\") pod \"cbade6b6-381e-4034-a3c7-075ce93c8218\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") "
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.421003    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-credential-socket\") pod \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") "
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.421039    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-data\") pod \"cbade6b6-381e-4034-a3c7-075ce93c8218\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") "
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.421080    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cbade6b6-381e-4034-a3c7-075ce93c8218-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbade6b6-381e-4034-a3c7-075ce93c8218-istiod-ca-cert\") pod \"cbade6b6-381e-4034-a3c7-075ce93c8218\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") "
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.421139    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-token\" (UniqueName: \"kubernetes.io/projected/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-token\") pod \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\" (UID: \"bbe524d9-e536-4990-ac0b-8ad75c9fd52e\") "
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.421174    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cbade6b6-381e-4034-a3c7-075ce93c8218-kube-api-access-vsrsj\" (UniqueName: \"kubernetes.io/projected/cbade6b6-381e-4034-a3c7-075ce93c8218-kube-api-access-vsrsj\") pod \"cbade6b6-381e-4034-a3c7-075ce93c8218\" (UID: \"cbade6b6-381e-4034-a3c7-075ce93c8218\") "
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.421182    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-workload-certs" pod "cbade6b6-381e-4034-a3c7-075ce93c8218" (UID: "cbade6b6-381e-4034-a3c7-075ce93c8218"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.421893    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-workload-socket" pod "bbe524d9-e536-4990-ac0b-8ad75c9fd52e" (UID: "bbe524d9-e536-4990-ac0b-8ad75c9fd52e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422017    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-workload-certs" pod "bbe524d9-e536-4990-ac0b-8ad75c9fd52e" (UID: "bbe524d9-e536-4990-ac0b-8ad75c9fd52e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422087    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422110    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422124    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422198    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422211    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422224    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422250    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23db1e9a-3c13-4c53-9c92-028efebcec23-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422287    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422304    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422305    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-credential-socket" pod "cbade6b6-381e-4034-a3c7-075ce93c8218" (UID: "cbade6b6-381e-4034-a3c7-075ce93c8218"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422318    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422331    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422345    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmkkx\" (UniqueName: \"kubernetes.io/projected/93475d2f-6f42-47e4-9c1e-44cd9080924c-kube-api-access-hmkkx\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422360    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422373    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93475d2f-6f42-47e4-9c1e-44cd9080924c-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.422391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422388    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.423829 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422425    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-workload-socket" pod "cbade6b6-381e-4034-a3c7-075ce93c8218" (UID: "cbade6b6-381e-4034-a3c7-075ce93c8218"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.423829 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422438    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23db1e9a-3c13-4c53-9c92-028efebcec23-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.423829 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422456    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rksnh\" (UniqueName: \"kubernetes.io/projected/23db1e9a-3c13-4c53-9c92-028efebcec23-kube-api-access-rksnh\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.423829 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422471    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93475d2f-6f42-47e4-9c1e-44cd9080924c-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.423829 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422536    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-data" pod "cbade6b6-381e-4034-a3c7-075ce93c8218" (UID: "cbade6b6-381e-4034-a3c7-075ce93c8218"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.423829 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422778    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-credential-socket" pod "bbe524d9-e536-4990-ac0b-8ad75c9fd52e" (UID: "bbe524d9-e536-4990-ac0b-8ad75c9fd52e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.423829 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422823    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbade6b6-381e-4034-a3c7-075ce93c8218-istiod-ca-cert" pod "cbade6b6-381e-4034-a3c7-075ce93c8218" (UID: "cbade6b6-381e-4034-a3c7-075ce93c8218"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:24.423829 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.422851    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istiod-ca-cert" pod "bbe524d9-e536-4990-ac0b-8ad75c9fd52e" (UID: "bbe524d9-e536-4990-ac0b-8ad75c9fd52e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:24.424802 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.424775    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-envoy" pod "bbe524d9-e536-4990-ac0b-8ad75c9fd52e" (UID: "bbe524d9-e536-4990-ac0b-8ad75c9fd52e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.424991 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.424955    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-kube-api-access-w46dq" pod "bbe524d9-e536-4990-ac0b-8ad75c9fd52e" (UID: "bbe524d9-e536-4990-ac0b-8ad75c9fd52e"). InnerVolumeSpecName "kube-api-access-w46dq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.425755 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.425728    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-token" pod "bbe524d9-e536-4990-ac0b-8ad75c9fd52e" (UID: "bbe524d9-e536-4990-ac0b-8ad75c9fd52e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.426037 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.425996    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbade6b6-381e-4034-a3c7-075ce93c8218-kube-api-access-vsrsj" pod "cbade6b6-381e-4034-a3c7-075ce93c8218" (UID: "cbade6b6-381e-4034-a3c7-075ce93c8218"). InnerVolumeSpecName "kube-api-access-vsrsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.426125 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.426095    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-token" pod "cbade6b6-381e-4034-a3c7-075ce93c8218" (UID: "cbade6b6-381e-4034-a3c7-075ce93c8218"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.426225 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.426208    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-podinfo" pod "cbade6b6-381e-4034-a3c7-075ce93c8218" (UID: "cbade6b6-381e-4034-a3c7-075ce93c8218"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:24.426653 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.426630    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-envoy" pod "cbade6b6-381e-4034-a3c7-075ce93c8218" (UID: "cbade6b6-381e-4034-a3c7-075ce93c8218"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.426753 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.426739    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-podinfo" pod "bbe524d9-e536-4990-ac0b-8ad75c9fd52e" (UID: "bbe524d9-e536-4990-ac0b-8ad75c9fd52e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:24.523304 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.523209    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-workload-certs\") pod \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") "
Oct 30 22:02:24.523486 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.523450    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-envoy\") pod \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") "
Oct 30 22:02:24.523573 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.523482    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d88dfc7e-36a5-4dd1-9814-2b966c89b085-kube-api-access-gjhtm\" (UniqueName: \"kubernetes.io/projected/d88dfc7e-36a5-4dd1-9814-2b966c89b085-kube-api-access-gjhtm\") pod \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") "
Oct 30 22:02:24.523573 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.523511    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-podinfo\") pod \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") "
Oct 30 22:02:24.523674 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.523599    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istiod-ca-cert\") pod \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") "
Oct 30 22:02:24.523674 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.523508    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-workload-certs" pod "d88dfc7e-36a5-4dd1-9814-2b966c89b085" (UID: "d88dfc7e-36a5-4dd1-9814-2b966c89b085"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.523774 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.523704    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-token\" (UniqueName: \"kubernetes.io/projected/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-token\") pod \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") "
Oct 30 22:02:24.523774 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.523757    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-credential-socket\") pod \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") "
Oct 30 22:02:24.523875 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.523812    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-workload-socket\") pod \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") "
Oct 30 22:02:24.523923 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.523909    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-data\") pod \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\" (UID: \"d88dfc7e-36a5-4dd1-9814-2b966c89b085\") "
Oct 30 22:02:24.524213 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524034    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istiod-ca-cert" pod "d88dfc7e-36a5-4dd1-9814-2b966c89b085" (UID: "d88dfc7e-36a5-4dd1-9814-2b966c89b085"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:24.524347 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524304    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-workload-socket" pod "d88dfc7e-36a5-4dd1-9814-2b966c89b085" (UID: "d88dfc7e-36a5-4dd1-9814-2b966c89b085"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.524492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524423    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-credential-socket" pod "d88dfc7e-36a5-4dd1-9814-2b966c89b085" (UID: "d88dfc7e-36a5-4dd1-9814-2b966c89b085"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.524652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524624    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-data" pod "d88dfc7e-36a5-4dd1-9814-2b966c89b085" (UID: "d88dfc7e-36a5-4dd1-9814-2b966c89b085"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.524652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524644    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.524834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524668    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w46dq\" (UniqueName: \"kubernetes.io/projected/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-kube-api-access-w46dq\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.524834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524686    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.524834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524701    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.524834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524719    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.524834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524734    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.524834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524750    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.524834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524763    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.524834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524777    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.524834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524791    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.524834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524805    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.524834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524823    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.524834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524836    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.525406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524851    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbade6b6-381e-4034-a3c7-075ce93c8218-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.525406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524864    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbade6b6-381e-4034-a3c7-075ce93c8218-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.525406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524880    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.525406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524895    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbe524d9-e536-4990-ac0b-8ad75c9fd52e-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.525406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524912    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsrsj\" (UniqueName: \"kubernetes.io/projected/cbade6b6-381e-4034-a3c7-075ce93c8218-kube-api-access-vsrsj\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.525406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.524926    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.526058 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.526034    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-podinfo" pod "d88dfc7e-36a5-4dd1-9814-2b966c89b085" (UID: "d88dfc7e-36a5-4dd1-9814-2b966c89b085"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:24.526103 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.526042    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-envoy" pod "d88dfc7e-36a5-4dd1-9814-2b966c89b085" (UID: "d88dfc7e-36a5-4dd1-9814-2b966c89b085"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:24.526320 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.526302    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d88dfc7e-36a5-4dd1-9814-2b966c89b085-kube-api-access-gjhtm" pod "d88dfc7e-36a5-4dd1-9814-2b966c89b085" (UID: "d88dfc7e-36a5-4dd1-9814-2b966c89b085"). InnerVolumeSpecName "kube-api-access-gjhtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.526380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.526332    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-token" pod "d88dfc7e-36a5-4dd1-9814-2b966c89b085" (UID: "d88dfc7e-36a5-4dd1-9814-2b966c89b085"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:24.625906 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.625869    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.625906 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.625901    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.625906 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.625912    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjhtm\" (UniqueName: \"kubernetes.io/projected/d88dfc7e-36a5-4dd1-9814-2b966c89b085-kube-api-access-gjhtm\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.626149 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.625921    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.626149 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.625931    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d88dfc7e-36a5-4dd1-9814-2b966c89b085-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:02:24.942758 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:24.942715    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:25.057935 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.057906    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"
Oct 30 22:02:25.058510 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.057904    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv" event={"ID":"bbe524d9-e536-4990-ac0b-8ad75c9fd52e","Type":"ContainerDied","Data":"9201fc5ae1d8cc818ad92af5af0f76191c924d2a43ae64d05dc5c82e90867f73"}
Oct 30 22:02:25.058635 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.058617    5586 scope.go:122] "RemoveContainer" containerID="667f837adaf4fe3b18d5f6d7ae00dd52971dfaf8de5cf2be9f354f3f7e40eaef"
Oct 30 22:02:25.059995 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.059914    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"
Oct 30 22:02:25.059995 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.059931    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n" event={"ID":"23db1e9a-3c13-4c53-9c92-028efebcec23","Type":"ContainerDied","Data":"cbcd033ff337cb835e4cd8e8f7fab19cb3597c8badc5020503316c186eb61211"}
Oct 30 22:02:25.061831 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.061807    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"
Oct 30 22:02:25.061952 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.061840    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr" event={"ID":"cbade6b6-381e-4034-a3c7-075ce93c8218","Type":"ContainerDied","Data":"e9949fe192a2e0f8aabd17b6ea84a4a72ca0574288726937f7e70140df5d8fb0"}
Oct 30 22:02:25.064680 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.064653    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w" event={"ID":"0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a","Type":"ContainerDied","Data":"52b89a6831aa88c6f33ab104d54f090fb292ee784d48b78d7a77d517799cc8e7"}
Oct 30 22:02:25.064814 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.064762    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"
Oct 30 22:02:25.081484 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.081443    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5" event={"ID":"93475d2f-6f42-47e4-9c1e-44cd9080924c","Type":"ContainerDied","Data":"027450e283f474e10c36d8e4716d7d8ab02b54719c45d95534d79fc2840d35a8"}
Oct 30 22:02:25.081614 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.081598    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"
Oct 30 22:02:25.084027 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.084003    5586 scope.go:122] "RemoveContainer" containerID="69aba28005014ce806931e1ebbb4cac213bf7b7ac086d9928236bad04c45cbc4"
Oct 30 22:02:25.087074 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.086996    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r" event={"ID":"c3360def-3e78-4997-98af-fefd89f483ee","Type":"ContainerDied","Data":"8a37fa4addf772829b514ee77da828bb264f905ec7d3e8f087d5e5b2f1f589ae"}
Oct 30 22:02:25.087212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.087189    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"
Oct 30 22:02:25.090329 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.090144    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"
Oct 30 22:02:25.093591 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.093502    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" event={"ID":"d88dfc7e-36a5-4dd1-9814-2b966c89b085","Type":"ContainerDied","Data":"eb0a29815889f8fc135577f2895a55e64b11838e6895a46876212323a35b708c"}
Oct 30 22:02:25.094130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.093910    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"
Oct 30 22:02:25.103332 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.102389    5586 scope.go:122] "RemoveContainer" containerID="d12812762a2d73874883ab158b5a51bdb53c24069a5ae1bf3e48beb524a1dcad"
Oct 30 22:02:25.116083 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.116054    5586 scope.go:122] "RemoveContainer" containerID="83c1121e575c1f56c2cb1e3c89536fed17a3828d8259b0ed045764d1e3d1b165"
Oct 30 22:02:25.128328 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.128301    5586 scope.go:122] "RemoveContainer" containerID="abe8dcf371ee3779ae62c82677519870e0d0a21a296f068db885f886606abf8c"
Oct 30 22:02:25.139891 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.139862    5586 scope.go:122] "RemoveContainer" containerID="2e9b09c76cae0dfc74829d6a4eaf9929437250dbc16b700bfe92e749d091ea5b"
Oct 30 22:02:25.151053 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.151033    5586 scope.go:122] "RemoveContainer" containerID="ec47de0245618ac8759325c15d25ca595a629c69f9b980867c547d94346b832b"
Oct 30 22:02:25.291602 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.291497    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"]
Oct 30 22:02:25.295662 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.295634    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-z5xnl"]
Oct 30 22:02:25.311323 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.311290    5586 patch_prober.go:29] interesting pod/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:02:25.311462 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.311356    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p" podUID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:02:25.403845 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.403810    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"]
Oct 30 22:02:25.415316 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.415277    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-v6wcr"]
Oct 30 22:02:25.448878 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.448845    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cbade6b6-381e-4034-a3c7-075ce93c8218" path="/var/lib/kubelet/pods/cbade6b6-381e-4034-a3c7-075ce93c8218/volumes"
Oct 30 22:02:25.449419 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.449401    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" path="/var/lib/kubelet/pods/ee1cab02-9630-4c82-b5ed-f3f00e4882ed/volumes"
Oct 30 22:02:25.522117 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.522081    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"]
Oct 30 22:02:25.526147 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.526121    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-4ls5w"]
Oct 30 22:02:25.632755 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.632670    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"]
Oct 30 22:02:25.641017 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.640914    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-x6q8r"]
Oct 30 22:02:25.753932 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.753900    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"]
Oct 30 22:02:25.757550 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.757526    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mxz6n"]
Oct 30 22:02:25.863444 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.863394    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"]
Oct 30 22:02:25.866987 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.866962    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-mv45p"]
Oct 30 22:02:25.974911 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.974877    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"]
Oct 30 22:02:25.981555 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:25.981518    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-9b51e1-openshift-default-6c7fc667bf-tfml5"]
Oct 30 22:02:26.090528 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:26.090486    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"]
Oct 30 22:02:26.094139 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:26.094110    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-dda9f5-openshift-default-59dc47f975-gv9rv"]
Oct 30 22:02:26.956711 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:26.956663    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:27.449920 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:27.449887    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" path="/var/lib/kubelet/pods/0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a/volumes"
Oct 30 22:02:27.450499 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:27.450482    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23db1e9a-3c13-4c53-9c92-028efebcec23" path="/var/lib/kubelet/pods/23db1e9a-3c13-4c53-9c92-028efebcec23/volumes"
Oct 30 22:02:27.450956 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:27.450942    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93475d2f-6f42-47e4-9c1e-44cd9080924c" path="/var/lib/kubelet/pods/93475d2f-6f42-47e4-9c1e-44cd9080924c/volumes"
Oct 30 22:02:27.451771 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:27.451749    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bbe524d9-e536-4990-ac0b-8ad75c9fd52e" path="/var/lib/kubelet/pods/bbe524d9-e536-4990-ac0b-8ad75c9fd52e/volumes"
Oct 30 22:02:27.452366 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:27.452344    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3360def-3e78-4997-98af-fefd89f483ee" path="/var/lib/kubelet/pods/c3360def-3e78-4997-98af-fefd89f483ee/volumes"
Oct 30 22:02:27.452964 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:27.452945    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" path="/var/lib/kubelet/pods/d88dfc7e-36a5-4dd1-9814-2b966c89b085/volumes"
Oct 30 22:02:27.937635 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:27.937596    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:28.179989 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.179951    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:28.341777 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.341674    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:28.567793 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.567760    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"]
Oct 30 22:02:28.568452 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568435    5586 cpu_manager.go:414] "containerMap: removing container" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy"
Oct 30 22:02:28.568452 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568453    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy"
Oct 30 22:02:28.568552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568469    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" containerName="istio-proxy"
Oct 30 22:02:28.568552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568475    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" containerName="istio-proxy"
Oct 30 22:02:28.568552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568487    5586 cpu_manager.go:414] "containerMap: removing container" podUID="0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" containerName="istio-proxy"
Oct 30 22:02:28.568552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568493    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" containerName="istio-proxy"
Oct 30 22:02:28.568552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568510    5586 cpu_manager.go:414] "containerMap: removing container" podUID="cbade6b6-381e-4034-a3c7-075ce93c8218" containerName="istio-proxy"
Oct 30 22:02:28.568552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568515    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cbade6b6-381e-4034-a3c7-075ce93c8218" containerName="istio-proxy"
Oct 30 22:02:28.568552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568525    5586 cpu_manager.go:414] "containerMap: removing container" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy"
Oct 30 22:02:28.568552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568529    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy"
Oct 30 22:02:28.568552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568547    5586 cpu_manager.go:414] "containerMap: removing container" podUID="bbe524d9-e536-4990-ac0b-8ad75c9fd52e" containerName="istio-proxy"
Oct 30 22:02:28.568552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568552    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbe524d9-e536-4990-ac0b-8ad75c9fd52e" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568560    5586 cpu_manager.go:414] "containerMap: removing container" podUID="23db1e9a-3c13-4c53-9c92-028efebcec23" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568564    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23db1e9a-3c13-4c53-9c92-028efebcec23" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568575    5586 cpu_manager.go:414] "containerMap: removing container" podUID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568579    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568585    5586 cpu_manager.go:414] "containerMap: removing container" podUID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568590    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568705    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="bbe524d9-e536-4990-ac0b-8ad75c9fd52e" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568714    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="23db1e9a-3c13-4c53-9c92-028efebcec23" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568722    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568727    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="93475d2f-6f42-47e4-9c1e-44cd9080924c" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568736    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3360def-3e78-4997-98af-fefd89f483ee" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568742    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d88dfc7e-36a5-4dd1-9814-2b966c89b085" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568747    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ff1f09e-67cd-4c5c-9831-75c9dd76aa5a" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568755    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="cbade6b6-381e-4034-a3c7-075ce93c8218" containerName="istio-proxy"
Oct 30 22:02:28.568846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.568761    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee1cab02-9630-4c82-b5ed-f3f00e4882ed" containerName="istio-proxy"
Oct 30 22:02:28.574484 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.574458    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"
Oct 30 22:02:28.613125 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.613028    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72v86\" (UniqueName: \"kubernetes.io/projected/b9c7ab71-5d38-4489-95fe-960b27c7bf40-kube-api-access-72v86\") pod \"payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4\" (UID: \"b9c7ab71-5d38-4489-95fe-960b27c7bf40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"
Oct 30 22:02:28.613319 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.613289    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9c7ab71-5d38-4489-95fe-960b27c7bf40-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4\" (UID: \"b9c7ab71-5d38-4489-95fe-960b27c7bf40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"
Oct 30 22:02:28.627608 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.627578    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-99f69c-dockercfg-f7wjt\""
Oct 30 22:02:28.628490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.628460    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-99f69c\""
Oct 30 22:02:28.640686 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.640101    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"]
Oct 30 22:02:28.735495 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.735431    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72v86\" (UniqueName: \"kubernetes.io/projected/b9c7ab71-5d38-4489-95fe-960b27c7bf40-kube-api-access-72v86\") pod \"payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4\" (UID: \"b9c7ab71-5d38-4489-95fe-960b27c7bf40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"
Oct 30 22:02:28.735764 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.735599    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9c7ab71-5d38-4489-95fe-960b27c7bf40-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4\" (UID: \"b9c7ab71-5d38-4489-95fe-960b27c7bf40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"
Oct 30 22:02:28.737025 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.736962    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9c7ab71-5d38-4489-95fe-960b27c7bf40-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4\" (UID: \"b9c7ab71-5d38-4489-95fe-960b27c7bf40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"
Oct 30 22:02:28.739758 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.739729    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"]
Oct 30 22:02:28.745540 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.745514    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"
Oct 30 22:02:28.799997 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.799958    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"]
Oct 30 22:02:28.801791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.800315    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72v86\" (UniqueName: \"kubernetes.io/projected/b9c7ab71-5d38-4489-95fe-960b27c7bf40-kube-api-access-72v86\") pod \"payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4\" (UID: \"b9c7ab71-5d38-4489-95fe-960b27c7bf40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"
Oct 30 22:02:28.838111 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.838061    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fgxr\" (UniqueName: \"kubernetes.io/projected/dd404a46-bc6b-4a63-bd7c-e09793a2e377-kube-api-access-5fgxr\") pod \"payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn\" (UID: \"dd404a46-bc6b-4a63-bd7c-e09793a2e377\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"
Oct 30 22:02:28.838418 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.838381    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd404a46-bc6b-4a63-bd7c-e09793a2e377-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn\" (UID: \"dd404a46-bc6b-4a63-bd7c-e09793a2e377\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"
Oct 30 22:02:28.893674 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.893496    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"
Oct 30 22:02:28.940465 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.940428    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fgxr\" (UniqueName: \"kubernetes.io/projected/dd404a46-bc6b-4a63-bd7c-e09793a2e377-kube-api-access-5fgxr\") pod \"payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn\" (UID: \"dd404a46-bc6b-4a63-bd7c-e09793a2e377\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"
Oct 30 22:02:28.940712 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.940684    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd404a46-bc6b-4a63-bd7c-e09793a2e377-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn\" (UID: \"dd404a46-bc6b-4a63-bd7c-e09793a2e377\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"
Oct 30 22:02:28.942094 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:28.942068    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd404a46-bc6b-4a63-bd7c-e09793a2e377-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn\" (UID: \"dd404a46-bc6b-4a63-bd7c-e09793a2e377\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"
Oct 30 22:02:29.000970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:29.000912    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fgxr\" (UniqueName: \"kubernetes.io/projected/dd404a46-bc6b-4a63-bd7c-e09793a2e377-kube-api-access-5fgxr\") pod \"payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn\" (UID: \"dd404a46-bc6b-4a63-bd7c-e09793a2e377\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"
Oct 30 22:02:29.070519 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:29.070474    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"
Oct 30 22:02:29.352863 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:29.352832    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"]
Oct 30 22:02:29.355607 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:02:29.355577    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9c7ab71_5d38_4489_95fe_960b27c7bf40.slice/crio-b59c574de20210a2e5b7de663044fbcb436e95a1a6ff92c86d953bad4818f940 WatchSource:0}: Error finding container b59c574de20210a2e5b7de663044fbcb436e95a1a6ff92c86d953bad4818f940: Status 404 returned error can't find the container with id b59c574de20210a2e5b7de663044fbcb436e95a1a6ff92c86d953bad4818f940
Oct 30 22:02:29.467918 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:29.467888    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"]
Oct 30 22:02:29.468783 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:02:29.468752    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd404a46_bc6b_4a63_bd7c_e09793a2e377.slice/crio-b49095181d4160ca23a9f754e7264a47f9f0711beb5585c976b9db2a577937d2 WatchSource:0}: Error finding container b49095181d4160ca23a9f754e7264a47f9f0711beb5585c976b9db2a577937d2: Status 404 returned error can't find the container with id b49095181d4160ca23a9f754e7264a47f9f0711beb5585c976b9db2a577937d2
Oct 30 22:02:29.673315 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:29.673207    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:29.843923 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:29.840673    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:30.146587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:30.146495    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn" event={"ID":"dd404a46-bc6b-4a63-bd7c-e09793a2e377","Type":"ContainerStarted","Data":"061a4484e6404e2697e164be52f5e6f2a54ef19e902663ef2923c9b592732d42"}
Oct 30 22:02:30.146587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:30.146583    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn" event={"ID":"dd404a46-bc6b-4a63-bd7c-e09793a2e377","Type":"ContainerStarted","Data":"b49095181d4160ca23a9f754e7264a47f9f0711beb5585c976b9db2a577937d2"}
Oct 30 22:02:30.146799 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:30.146608    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"
Oct 30 22:02:30.148630 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:30.148596    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4" event={"ID":"b9c7ab71-5d38-4489-95fe-960b27c7bf40","Type":"ContainerStarted","Data":"b1663a041010bb706780dfdaa3d1445b8f1f528b7f0f791140a61ae3f99b7e35"}
Oct 30 22:02:30.148766 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:30.148647    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4" event={"ID":"b9c7ab71-5d38-4489-95fe-960b27c7bf40","Type":"ContainerStarted","Data":"b59c574de20210a2e5b7de663044fbcb436e95a1a6ff92c86d953bad4818f940"}
Oct 30 22:02:30.148766 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:30.148699    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"
Oct 30 22:02:30.261297 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:30.261242    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn" podStartSLOduration=2.261216798 podStartE2EDuration="2.261216798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:30.259361905 +0000 UTC m=+1692.931265840" watchObservedRunningTime="2026-10-30 22:02:30.261216798 +0000 UTC m=+1692.933120733"
Oct 30 22:02:30.384245 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:30.384170    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4" podStartSLOduration=2.384151986 podStartE2EDuration="2.384151986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:30.383677716 +0000 UTC m=+1693.055581651" watchObservedRunningTime="2026-10-30 22:02:30.384151986 +0000 UTC m=+1693.056055925"
Oct 30 22:02:32.947000 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:32.946961    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"]
Oct 30 22:02:32.951608 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:32.951578    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:33.004361 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.004322    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e2deec91-serving-cert\""
Oct 30 22:02:33.015329 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.015289    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"]
Oct 30 22:02:33.130471 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.130438    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-metrics-certs\") pod \"maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:33.130639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.130494    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-maas-api-tls\") pod \"maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:33.130639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.130514    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1baa2c91-37cd-4e6c-a85d-8674138b03fe-openshift-service-ca\") pod \"maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:33.130770 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.130747    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm48d\" (UniqueName: \"kubernetes.io/projected/1baa2c91-37cd-4e6c-a85d-8674138b03fe-kube-api-access-nm48d\") pod \"maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:33.206702 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.206604    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:33.232082 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.232042    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-metrics-certs\") pod \"maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:33.232332 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.232106    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-maas-api-tls\") pod \"maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:33.232332 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.232125    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1baa2c91-37cd-4e6c-a85d-8674138b03fe-openshift-service-ca\") pod \"maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:33.232332 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.232296    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm48d\" (UniqueName: \"kubernetes.io/projected/1baa2c91-37cd-4e6c-a85d-8674138b03fe-kube-api-access-nm48d\") pod \"maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:33.232332 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:02:33.232305    5586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-serving-cert: secret "maas-api-e2e-mt-e2deec91-serving-cert" not found
Oct 30 22:02:33.232556 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:02:33.232383    5586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-maas-api-tls podName:1baa2c91-37cd-4e6c-a85d-8674138b03fe nodeName:}" failed. No retries permitted until 2026-10-30 22:02:33.732360728 +0000 UTC m=+1696.404264649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-maas-api-tls") pod "maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r" (UID: "1baa2c91-37cd-4e6c-a85d-8674138b03fe") : secret "maas-api-e2e-mt-e2deec91-serving-cert" not found
Oct 30 22:02:33.232945 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.232926    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1baa2c91-37cd-4e6c-a85d-8674138b03fe-openshift-service-ca\") pod \"maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:33.234716 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.234691    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-metrics-certs\") pod \"maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:33.290417 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.290371    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm48d\" (UniqueName: \"kubernetes.io/projected/1baa2c91-37cd-4e6c-a85d-8674138b03fe-kube-api-access-nm48d\") pod \"maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:33.421369 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.421328    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:33.740631 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.740588    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-maas-api-tls\") pod \"maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:33.743144 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.743116    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-maas-api-tls\") pod \"maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:33.819871 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.819834    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:33.866786 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:33.866750    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:34.256982 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:34.256946    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"]
Oct 30 22:02:34.258134 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:02:34.258089    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1baa2c91_37cd_4e6c_a85d_8674138b03fe.slice/crio-5c524c40ebf10828fe72eb1f6cabbb3f93fea89f7e2131e28075fdd688a454de WatchSource:0}: Error finding container 5c524c40ebf10828fe72eb1f6cabbb3f93fea89f7e2131e28075fdd688a454de: Status 404 returned error can't find the container with id 5c524c40ebf10828fe72eb1f6cabbb3f93fea89f7e2131e28075fdd688a454de
Oct 30 22:02:34.259740 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:34.259722    5586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:02:34.559160 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:34.559065    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:35.213350 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.213294    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r" event={"ID":"1baa2c91-37cd-4e6c-a85d-8674138b03fe","Type":"ContainerStarted","Data":"ffbbc35e9d21818a4d21b4c3ceece5d98fdabf59ffb8dc320ceef26dbdefaad6"}
Oct 30 22:02:35.213708 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.213676    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r" event={"ID":"1baa2c91-37cd-4e6c-a85d-8674138b03fe","Type":"ContainerStarted","Data":"5c524c40ebf10828fe72eb1f6cabbb3f93fea89f7e2131e28075fdd688a454de"}
Oct 30 22:02:35.215101 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.215079    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:35.889969 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.889910    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r" podStartSLOduration=3.069862362 podStartE2EDuration="3.88989076s" totalImagesPullingTime="820.028398ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:35.340410302 +0000 UTC m=+1698.012314238" watchObservedRunningTime="2026-10-30 22:02:35.88989076 +0000 UTC m=+1698.561794696"
Oct 30 22:02:35.892079 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.892055    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"]
Oct 30 22:02:35.896637 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.896613    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:35.961011 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.960975    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"]
Oct 30 22:02:35.983860 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.983804    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:35.984077 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.983871    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:35.984077 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.983913    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv94v\" (UniqueName: \"kubernetes.io/projected/8a89955b-47d7-4a3a-895f-6c3f69b695aa-kube-api-access-bv94v\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:35.984077 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.983998    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:35.984077 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.984074    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:35.984333 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.984104    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:35.984333 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.984139    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:35.984333 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.984269    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:35.984333 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.984296    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:35.999595 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:35.999545    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6df3ba0e-openshift-default-dockercfg-6ckx9\""
Oct 30 22:02:36.011315 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.011249    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:36.085152 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.085112    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.085379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.085169    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.085379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.085210    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv94v\" (UniqueName: \"kubernetes.io/projected/8a89955b-47d7-4a3a-895f-6c3f69b695aa-kube-api-access-bv94v\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.085379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.085270    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.085379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.085342    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.085379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.085370    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.085647 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.085620    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.085931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.085895    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.086038 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.085981    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.086123 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.086096    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.086331 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.086148    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.087798 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.086537    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.087798 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.086743    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.087798 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.087209    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.088558 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.088535    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.089433 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.089408    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.142025 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.141948    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.142278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.142250    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv94v\" (UniqueName: \"kubernetes.io/projected/8a89955b-47d7-4a3a-895f-6c3f69b695aa-kube-api-access-bv94v\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.240259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.225950    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:36.240259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.229382    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:36.734821 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.734785    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"]
Oct 30 22:02:36.736133 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:02:36.736105    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a89955b_47d7_4a3a_895f_6c3f69b695aa.slice/crio-df37d5d10d84443ece8133aed532c63771332bba20ff286382f614671e97c07c WatchSource:0}: Error finding container df37d5d10d84443ece8133aed532c63771332bba20ff286382f614671e97c07c: Status 404 returned error can't find the container with id df37d5d10d84443ece8133aed532c63771332bba20ff286382f614671e97c07c
Oct 30 22:02:36.738889 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.738856    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:02:36.738956 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.738928    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:02:36.739164 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:36.738956    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:02:37.289794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:37.289705    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" event={"ID":"8a89955b-47d7-4a3a-895f-6c3f69b695aa","Type":"ContainerStarted","Data":"d525a8b9d7cfa3cd8d201500ae8112c65218a8b7eb984ccb8324448ae8d0cdab"}
Oct 30 22:02:37.289794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:37.289792    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" event={"ID":"8a89955b-47d7-4a3a-895f-6c3f69b695aa","Type":"ContainerStarted","Data":"df37d5d10d84443ece8133aed532c63771332bba20ff286382f614671e97c07c"}
Oct 30 22:02:37.423078 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:37.423020    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" podStartSLOduration=2.42300524 podStartE2EDuration="2.42300524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:37.419045006 +0000 UTC m=+1700.090948941" watchObservedRunningTime="2026-10-30 22:02:37.42300524 +0000 UTC m=+1700.094909175"
Oct 30 22:02:38.042554 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.042514    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"]
Oct 30 22:02:38.048373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.048346    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:38.103642 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.103608    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"]
Oct 30 22:02:38.125268 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.125212    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nl79\" (UniqueName: \"kubernetes.io/projected/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-kube-api-access-2nl79\") pod \"maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:38.125667 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.125634    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:38.125800 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.125705    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:38.125976 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.125951    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:38.148212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.148174    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-326d76-serving-cert\""
Oct 30 22:02:38.227611 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.227573    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:38.227834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.227816    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:38.227965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.227949    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nl79\" (UniqueName: \"kubernetes.io/projected/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-kube-api-access-2nl79\") pod \"maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:38.228043 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.227983    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:38.228139 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:02:38.228123    5586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-serving-cert: secret "maas-api-e2e-shared-a-w3-326d76-serving-cert" not found
Oct 30 22:02:38.228221 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:02:38.228209    5586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-maas-api-tls podName:d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf nodeName:}" failed. No retries permitted until 2026-10-30 22:02:38.728188953 +0000 UTC m=+1701.400092887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-maas-api-tls") pod "maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp" (UID: "d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf") : secret "maas-api-e2e-shared-a-w3-326d76-serving-cert" not found
Oct 30 22:02:38.228422 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.228402    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:38.230343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.230315    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:38.232288 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.232226    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:38.238343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.238320    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:38.262285 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.262219    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:38.323821 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.323704    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nl79\" (UniqueName: \"kubernetes.io/projected/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-kube-api-access-2nl79\") pod \"maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:38.329310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.329284    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:38.330805 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.330776    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:02:38.454149 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.454110    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:38.628185 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.628082    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:38.738652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.738620    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:38.741216 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.741189    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:38.965289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:38.965219    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:39.418256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:39.397699    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"]
Oct 30 22:02:40.397819 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:40.397785    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp" event={"ID":"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf","Type":"ContainerStarted","Data":"c39abb58ece5a239565daa98b508721abe9a91ccdaa9a76e32059a6ae0310c3c"}
Oct 30 22:02:40.690596 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:40.690558    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:40.901931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:40.901891    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:41.160544 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:41.160515    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"
Oct 30 22:02:41.160725 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:41.160591    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"
Oct 30 22:02:41.565845 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:41.565668    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:41.565845 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:41.565750    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp" event={"ID":"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf","Type":"ContainerStarted","Data":"5126a1904e84c2472870d8088258c4c5ac8c7d6aa1f23fa9e64306c89b5fade6"}
Oct 30 22:02:41.682308 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:41.682210    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp" podStartSLOduration=3.769111594 podStartE2EDuration="4.682188745s" totalImagesPullingTime="913.077151ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:41.681228221 +0000 UTC m=+1704.353132157" watchObservedRunningTime="2026-10-30 22:02:41.682188745 +0000 UTC m=+1704.354092697"
Oct 30 22:02:42.297274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:42.297208    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:02:42.638344 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:42.636309    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:43.836897 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:43.836854    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:44.027606 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.027570    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:44.225526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.225491    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:44.448093 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.448056    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"]
Oct 30 22:02:44.452225 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.452201    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"
Oct 30 22:02:44.471591 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.471553    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ec4ead0-ee39-4c04-8512-716f6143b147-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll\" (UID: \"4ec4ead0-ee39-4c04-8512-716f6143b147\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"
Oct 30 22:02:44.471812 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.471793    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw2v9\" (UniqueName: \"kubernetes.io/projected/4ec4ead0-ee39-4c04-8512-716f6143b147-kube-api-access-jw2v9\") pod \"payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll\" (UID: \"4ec4ead0-ee39-4c04-8512-716f6143b147\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"
Oct 30 22:02:44.592313 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.589557    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-6df3ba0e\""
Oct 30 22:02:44.606257 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.603836    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-6df3ba0e-dockercfg-csrlc\""
Oct 30 22:02:44.606257 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.604592    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw2v9\" (UniqueName: \"kubernetes.io/projected/4ec4ead0-ee39-4c04-8512-716f6143b147-kube-api-access-jw2v9\") pod \"payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll\" (UID: \"4ec4ead0-ee39-4c04-8512-716f6143b147\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"
Oct 30 22:02:44.606257 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.604962    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ec4ead0-ee39-4c04-8512-716f6143b147-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll\" (UID: \"4ec4ead0-ee39-4c04-8512-716f6143b147\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"
Oct 30 22:02:44.678254 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.674640    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ec4ead0-ee39-4c04-8512-716f6143b147-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll\" (UID: \"4ec4ead0-ee39-4c04-8512-716f6143b147\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"
Oct 30 22:02:44.703254 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.703191    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"]
Oct 30 22:02:44.757713 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.757677    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"]
Oct 30 22:02:44.762561 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.762289    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw2v9\" (UniqueName: \"kubernetes.io/projected/4ec4ead0-ee39-4c04-8512-716f6143b147-kube-api-access-jw2v9\") pod \"payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll\" (UID: \"4ec4ead0-ee39-4c04-8512-716f6143b147\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"
Oct 30 22:02:44.766936 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.766661    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"
Oct 30 22:02:44.770638 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.770611    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"]
Oct 30 22:02:44.770795 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.770737    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"
Oct 30 22:02:44.831839 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.831551    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b8ea9ca-9405-4abf-852c-04a8289fb6ec-plugins-config-volume\") pod \"payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n\" (UID: \"7b8ea9ca-9405-4abf-852c-04a8289fb6ec\") " pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"
Oct 30 22:02:44.831839 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.831703    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74x8z\" (UniqueName: \"kubernetes.io/projected/7b8ea9ca-9405-4abf-852c-04a8289fb6ec-kube-api-access-74x8z\") pod \"payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n\" (UID: \"7b8ea9ca-9405-4abf-852c-04a8289fb6ec\") " pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"
Oct 30 22:02:44.833714 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.833687    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:44.935389 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.935344    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b8ea9ca-9405-4abf-852c-04a8289fb6ec-plugins-config-volume\") pod \"payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n\" (UID: \"7b8ea9ca-9405-4abf-852c-04a8289fb6ec\") " pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"
Oct 30 22:02:44.935808 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.935498    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74x8z\" (UniqueName: \"kubernetes.io/projected/7b8ea9ca-9405-4abf-852c-04a8289fb6ec-kube-api-access-74x8z\") pod \"payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n\" (UID: \"7b8ea9ca-9405-4abf-852c-04a8289fb6ec\") " pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"
Oct 30 22:02:44.936080 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.936059    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b8ea9ca-9405-4abf-852c-04a8289fb6ec-plugins-config-volume\") pod \"payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n\" (UID: \"7b8ea9ca-9405-4abf-852c-04a8289fb6ec\") " pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"
Oct 30 22:02:44.992085 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:44.992055    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74x8z\" (UniqueName: \"kubernetes.io/projected/7b8ea9ca-9405-4abf-852c-04a8289fb6ec-kube-api-access-74x8z\") pod \"payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n\" (UID: \"7b8ea9ca-9405-4abf-852c-04a8289fb6ec\") " pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"
Oct 30 22:02:45.137821 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:45.137723    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"
Oct 30 22:02:45.156732 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:45.156703    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"]
Oct 30 22:02:45.158163 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:02:45.158130    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ec4ead0_ee39_4c04_8512_716f6143b147.slice/crio-32f0502a3fac68ef3177d7f13db703331f0602bbad06ef498128715165ea6bea WatchSource:0}: Error finding container 32f0502a3fac68ef3177d7f13db703331f0602bbad06ef498128715165ea6bea: Status 404 returned error can't find the container with id 32f0502a3fac68ef3177d7f13db703331f0602bbad06ef498128715165ea6bea
Oct 30 22:02:45.539720 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:45.539684    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"]
Oct 30 22:02:45.540578 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:02:45.540556    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b8ea9ca_9405_4abf_852c_04a8289fb6ec.slice/crio-e599fa39a655adc7ce92d920f0617f022bd6e1e946d7209e056c201a151a2a04 WatchSource:0}: Error finding container e599fa39a655adc7ce92d920f0617f022bd6e1e946d7209e056c201a151a2a04: Status 404 returned error can't find the container with id e599fa39a655adc7ce92d920f0617f022bd6e1e946d7209e056c201a151a2a04
Oct 30 22:02:45.771012 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:45.770975    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n" event={"ID":"7b8ea9ca-9405-4abf-852c-04a8289fb6ec","Type":"ContainerStarted","Data":"9b1c0634616f5de3eac94e79297aee28b25d177d5a528a65521c7349f98ac2d5"}
Oct 30 22:02:45.771223 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:45.771037    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n" event={"ID":"7b8ea9ca-9405-4abf-852c-04a8289fb6ec","Type":"ContainerStarted","Data":"e599fa39a655adc7ce92d920f0617f022bd6e1e946d7209e056c201a151a2a04"}
Oct 30 22:02:45.771223 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:45.771063    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"
Oct 30 22:02:45.773293 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:45.773262    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll" event={"ID":"4ec4ead0-ee39-4c04-8512-716f6143b147","Type":"ContainerStarted","Data":"cb751c94cf26aa6b65e3b19f7baf4e79d9e698a6e0447f20c227dd1f92dd45bb"}
Oct 30 22:02:45.773414 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:45.773321    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll" event={"ID":"4ec4ead0-ee39-4c04-8512-716f6143b147","Type":"ContainerStarted","Data":"32f0502a3fac68ef3177d7f13db703331f0602bbad06ef498128715165ea6bea"}
Oct 30 22:02:45.773414 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:45.773345    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"
Oct 30 22:02:45.906623 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:45.906573    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n" podStartSLOduration=1.906560163 podStartE2EDuration="1.906560163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:45.906217744 +0000 UTC m=+1708.578121681" watchObservedRunningTime="2026-10-30 22:02:45.906560163 +0000 UTC m=+1708.578464099"
Oct 30 22:02:46.031970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:46.031903    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll" podStartSLOduration=2.031883996 podStartE2EDuration="2.031883996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:46.028631375 +0000 UTC m=+1708.700535311" watchObservedRunningTime="2026-10-30 22:02:46.031883996 +0000 UTC m=+1708.703787932"
Oct 30 22:02:47.576044 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:47.576015    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:02:48.362627 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:02:48.362584    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:02:48.753624 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:48.753590    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:48.969485 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:48.969447    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:53.782851 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:53.782804    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:56.784989 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:56.784958    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"
Oct 30 22:02:56.785418 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:56.785030    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"
Oct 30 22:02:57.875477 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:57.875431    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:59.503049 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:59.502932    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:59.677322 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:59.677284    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:02:59.828143 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:02:59.828061    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:00.192157 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:00.192114    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:01.108008 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.107971    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:01.639517 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.639485    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"]
Oct 30 22:03:01.645269 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.645228    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.706992 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.706944    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-token\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.707207 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.707113    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istiod-ca-cert\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.707207 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.707185    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-credential-socket\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.707402 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.707299    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-workload-socket\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.707402 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.707347    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-data\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.707522 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.707423    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-envoy\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.707522 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.707473    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-workload-certs\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.707629 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.707533    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqpjq\" (UniqueName: \"kubernetes.io/projected/e1307e68-d9b4-48e0-bad6-44854d56a0c4-kube-api-access-tqpjq\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.707726 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.707698    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-podinfo\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.712701 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.712660    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-ea3cb2-openshift-default-dockercfg-zm7ld\""
Oct 30 22:03:01.748479 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.748443    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"]
Oct 30 22:03:01.749223 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.749189    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:01.809304 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.809259    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-data\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.809503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.809313    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-envoy\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.809503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.809351    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-workload-certs\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.809503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.809381    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqpjq\" (UniqueName: \"kubernetes.io/projected/e1307e68-d9b4-48e0-bad6-44854d56a0c4-kube-api-access-tqpjq\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.809503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.809487    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-podinfo\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.809742 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.809622    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-token\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.809742 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.809698    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istiod-ca-cert\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.809742 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.809699    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-data\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.809914 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.809771    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-credential-socket\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.809914 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.809807    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-workload-socket\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.810160 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.810140    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-credential-socket\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.810313 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.810263    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-workload-certs\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.810313 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.810274    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-workload-socket\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.810464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.810445    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istiod-ca-cert\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.812152 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.812124    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-podinfo\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.812268 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.812154    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-envoy\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.866090 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.866052    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-token\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.866090 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.866088    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqpjq\" (UniqueName: \"kubernetes.io/projected/e1307e68-d9b4-48e0-bad6-44854d56a0c4-kube-api-access-tqpjq\") pod \"e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") " pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:01.961021 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:01.960978    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:02.010674 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:02.010636    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:02.411164 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:02.411132    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"]
Oct 30 22:03:02.414889 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:03:02.414855    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1307e68_d9b4_48e0_bad6_44854d56a0c4.slice/crio-89312ea12fc42ab8fc4a84cb20eb3a00e75e102082f2302e143f74f0f2e1280f WatchSource:0}: Error finding container 89312ea12fc42ab8fc4a84cb20eb3a00e75e102082f2302e143f74f0f2e1280f: Status 404 returned error can't find the container with id 89312ea12fc42ab8fc4a84cb20eb3a00e75e102082f2302e143f74f0f2e1280f
Oct 30 22:03:02.417699 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:02.417668    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:02.417770 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:02.417733    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:02.417770 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:02.417764    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:02.950809 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:02.950764    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" event={"ID":"e1307e68-d9b4-48e0-bad6-44854d56a0c4","Type":"ContainerStarted","Data":"190677165c88e636d400ced972849c243e1c36f772db8fde1702cb9a3caa88a7"}
Oct 30 22:03:02.950991 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:02.950829    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" event={"ID":"e1307e68-d9b4-48e0-bad6-44854d56a0c4","Type":"ContainerStarted","Data":"89312ea12fc42ab8fc4a84cb20eb3a00e75e102082f2302e143f74f0f2e1280f"}
Oct 30 22:03:02.961330 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:02.961283    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:03.056390 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:03.056352    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"]
Oct 30 22:03:03.056639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:03.056598    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" containerID="cri-o://29b68d69c49d91bf30607ad98540ddb3439012f394ada62d82db52543455229b" gracePeriod=30
Oct 30 22:03:03.061587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:03.061561    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"]
Oct 30 22:03:03.061758 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:03.061738    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" containerID="cri-o://de84582f74c1b9b6cf0e326d08d3fa53c47e6e83113534ca0ec59e30e2d31fb0" gracePeriod=30
Oct 30 22:03:03.065320 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:03.065215    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"]
Oct 30 22:03:03.065560 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:03.065510    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd" podUID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerName="istio-proxy" containerID="cri-o://49626e30c184a07ec9f242c441f2cad15c993f1b4e120cf51fafa38ac77ad15c" gracePeriod=30
Oct 30 22:03:03.069858 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:03.069830    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"]
Oct 30 22:03:03.070049 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:03.070025    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" podUID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" containerName="istio-proxy" containerID="cri-o://c13342f0489dbdc044381f97127790e3f503c4c978f4a7c77d58a31d1247eff8" gracePeriod=30
Oct 30 22:03:03.103326 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:03.103270    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" podStartSLOduration=2.103252162 podStartE2EDuration="2.103252162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:03.102148212 +0000 UTC m=+1725.774052147" watchObservedRunningTime="2026-10-30 22:03:03.103252162 +0000 UTC m=+1725.775156114"
Oct 30 22:03:03.940636 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:03.940596    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:03:03.941047 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:03.940665    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:03:03.968225 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:03.968194    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:04.521996 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:04.521949    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:04.970362 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:04.970330    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:04.971581 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:04.971558    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:03:05.006613 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:05.006564    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:03:05.006801 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:05.006648    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:03:06.181825 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:06.181791    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:06.462168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:06.462079    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:06.617166 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:06.617133    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:06.821226 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:06.821131    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:07.127474 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:07.127385    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:07.344766 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:07.344733    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:07.507580 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:07.507537    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:07.737766 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:07.737729    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:08.258363 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.249982    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:08.429340 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.401006    5586 patch_prober.go:29] interesting pod/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:03:08.429340 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.401063    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" podUID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 22:03:08.472752 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.472115    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:08.574202 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.573344    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 22:03:08.627452 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.627421    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-credential-socket\") pod \"c145c160-382f-4151-bfd1-cab6a9e4dc98\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") "
Oct 30 22:03:08.627685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.627483    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-token\" (UniqueName: \"kubernetes.io/projected/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-token\") pod \"c145c160-382f-4151-bfd1-cab6a9e4dc98\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") "
Oct 30 22:03:08.627685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.627522    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c145c160-382f-4151-bfd1-cab6a9e4dc98-kube-api-access-l2s62\" (UniqueName: \"kubernetes.io/projected/c145c160-382f-4151-bfd1-cab6a9e4dc98-kube-api-access-l2s62\") pod \"c145c160-382f-4151-bfd1-cab6a9e4dc98\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") "
Oct 30 22:03:08.627685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.627557    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-envoy\") pod \"c145c160-382f-4151-bfd1-cab6a9e4dc98\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") "
Oct 30 22:03:08.627685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.627611    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c145c160-382f-4151-bfd1-cab6a9e4dc98-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c145c160-382f-4151-bfd1-cab6a9e4dc98-istiod-ca-cert\") pod \"c145c160-382f-4151-bfd1-cab6a9e4dc98\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") "
Oct 30 22:03:08.627685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.627682    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-workload-certs\") pod \"c145c160-382f-4151-bfd1-cab6a9e4dc98\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") "
Oct 30 22:03:08.628110 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.627791    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-workload-socket\") pod \"c145c160-382f-4151-bfd1-cab6a9e4dc98\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") "
Oct 30 22:03:08.628110 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.627830    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-podinfo\") pod \"c145c160-382f-4151-bfd1-cab6a9e4dc98\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") "
Oct 30 22:03:08.628110 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.627906    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-data\") pod \"c145c160-382f-4151-bfd1-cab6a9e4dc98\" (UID: \"c145c160-382f-4151-bfd1-cab6a9e4dc98\") "
Oct 30 22:03:08.629055 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.629026    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-data" pod "c145c160-382f-4151-bfd1-cab6a9e4dc98" (UID: "c145c160-382f-4151-bfd1-cab6a9e4dc98"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.629362 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.629341    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-credential-socket" pod "c145c160-382f-4151-bfd1-cab6a9e4dc98" (UID: "c145c160-382f-4151-bfd1-cab6a9e4dc98"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.630256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.629978    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c145c160-382f-4151-bfd1-cab6a9e4dc98-istiod-ca-cert" pod "c145c160-382f-4151-bfd1-cab6a9e4dc98" (UID: "c145c160-382f-4151-bfd1-cab6a9e4dc98"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:08.630256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.630210    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-workload-certs" pod "c145c160-382f-4151-bfd1-cab6a9e4dc98" (UID: "c145c160-382f-4151-bfd1-cab6a9e4dc98"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.630692 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.630661    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-workload-socket" pod "c145c160-382f-4151-bfd1-cab6a9e4dc98" (UID: "c145c160-382f-4151-bfd1-cab6a9e4dc98"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.633514 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.633489    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-token" pod "c145c160-382f-4151-bfd1-cab6a9e4dc98" (UID: "c145c160-382f-4151-bfd1-cab6a9e4dc98"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:08.633627 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.633608    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c145c160-382f-4151-bfd1-cab6a9e4dc98-kube-api-access-l2s62" pod "c145c160-382f-4151-bfd1-cab6a9e4dc98" (UID: "c145c160-382f-4151-bfd1-cab6a9e4dc98"). InnerVolumeSpecName "kube-api-access-l2s62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:08.648074 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.648018    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-podinfo" pod "c145c160-382f-4151-bfd1-cab6a9e4dc98" (UID: "c145c160-382f-4151-bfd1-cab6a9e4dc98"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:03:08.653791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.653757    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-envoy" pod "c145c160-382f-4151-bfd1-cab6a9e4dc98" (UID: "c145c160-382f-4151-bfd1-cab6a9e4dc98"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.687775 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.687751    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 22:03:08.689707 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.689688    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 22:03:08.693334 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.693314    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 22:03:08.729805 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.729768    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-envoy\") pod \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") "
Oct 30 22:03:08.729989 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.729842    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-workload-certs\") pod \"d9f74a60-e9aa-4366-9c25-891da9aee43c\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") "
Oct 30 22:03:08.729989 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.729879    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-token\" (UniqueName: \"kubernetes.io/projected/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-token\") pod \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") "
Oct 30 22:03:08.729989 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.729922    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-envoy\") pod \"d9f74a60-e9aa-4366-9c25-891da9aee43c\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") "
Oct 30 22:03:08.729989 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.729950    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-podinfo\") pod \"d9f74a60-e9aa-4366-9c25-891da9aee43c\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") "
Oct 30 22:03:08.730298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.729992    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-credential-socket\") pod \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") "
Oct 30 22:03:08.730298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730021    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-token\" (UniqueName: \"kubernetes.io/projected/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-token\") pod \"d9f74a60-e9aa-4366-9c25-891da9aee43c\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") "
Oct 30 22:03:08.730298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730050    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-workload-socket\") pod \"d9f74a60-e9aa-4366-9c25-891da9aee43c\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") "
Oct 30 22:03:08.730298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730105    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-workload-socket\") pod \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") "
Oct 30 22:03:08.730298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730140    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-podinfo\") pod \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") "
Oct 30 22:03:08.730298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730185    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-envoy\") pod \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") "
Oct 30 22:03:08.730298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730248    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istiod-ca-cert\") pod \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") "
Oct 30 22:03:08.730298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730293    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-workload-socket\") pod \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") "
Oct 30 22:03:08.730749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730366    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-token\" (UniqueName: \"kubernetes.io/projected/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-token\") pod \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") "
Oct 30 22:03:08.730749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730403    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-credential-socket\") pod \"d9f74a60-e9aa-4366-9c25-891da9aee43c\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") "
Oct 30 22:03:08.730749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730444    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9f74a60-e9aa-4366-9c25-891da9aee43c-kube-api-access-w7gr8\" (UniqueName: \"kubernetes.io/projected/d9f74a60-e9aa-4366-9c25-891da9aee43c-kube-api-access-w7gr8\") pod \"d9f74a60-e9aa-4366-9c25-891da9aee43c\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") "
Oct 30 22:03:08.730749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730495    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-data\") pod \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") "
Oct 30 22:03:08.730749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730531    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-data\") pod \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") "
Oct 30 22:03:08.730749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730567    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-podinfo\") pod \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") "
Oct 30 22:03:08.730749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730731    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-kube-api-access-68fz6\" (UniqueName: \"kubernetes.io/projected/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-kube-api-access-68fz6\") pod \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") "
Oct 30 22:03:08.731959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730748    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-credential-socket" pod "5568d46b-ed6b-4c97-99e0-8a0de7cc5258" (UID: "5568d46b-ed6b-4c97-99e0-8a0de7cc5258"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.731959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730772    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd9299f1-9812-4f6c-8851-6957b8b3ab64-kube-api-access-s47tj\" (UniqueName: \"kubernetes.io/projected/cd9299f1-9812-4f6c-8851-6957b8b3ab64-kube-api-access-s47tj\") pod \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") "
Oct 30 22:03:08.731959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730869    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-workload-certs\") pod \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") "
Oct 30 22:03:08.731959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.730919    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-credential-socket\") pod \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") "
Oct 30 22:03:08.731959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.731049    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-workload-certs" pod "d9f74a60-e9aa-4366-9c25-891da9aee43c" (UID: "d9f74a60-e9aa-4366-9c25-891da9aee43c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.731959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.731071    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istiod-ca-cert\") pod \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\" (UID: \"cd9299f1-9812-4f6c-8851-6957b8b3ab64\") "
Oct 30 22:03:08.731959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.731153    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-workload-certs\") pod \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\" (UID: \"5568d46b-ed6b-4c97-99e0-8a0de7cc5258\") "
Oct 30 22:03:08.731959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.731212    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9f74a60-e9aa-4366-9c25-891da9aee43c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9f74a60-e9aa-4366-9c25-891da9aee43c-istiod-ca-cert\") pod \"d9f74a60-e9aa-4366-9c25-891da9aee43c\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") "
Oct 30 22:03:08.731959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.731217    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-workload-socket" pod "5568d46b-ed6b-4c97-99e0-8a0de7cc5258" (UID: "5568d46b-ed6b-4c97-99e0-8a0de7cc5258"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.731959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.731476    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-credential-socket" pod "cd9299f1-9812-4f6c-8851-6957b8b3ab64" (UID: "cd9299f1-9812-4f6c-8851-6957b8b3ab64"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.731959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.731686    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-workload-certs" pod "cd9299f1-9812-4f6c-8851-6957b8b3ab64" (UID: "cd9299f1-9812-4f6c-8851-6957b8b3ab64"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.733111 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.733080    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-credential-socket" pod "d9f74a60-e9aa-4366-9c25-891da9aee43c" (UID: "d9f74a60-e9aa-4366-9c25-891da9aee43c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.733994 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.732359    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-data" pod "5568d46b-ed6b-4c97-99e0-8a0de7cc5258" (UID: "5568d46b-ed6b-4c97-99e0-8a0de7cc5258"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.733994 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.733486    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-workload-socket" pod "d9f74a60-e9aa-4366-9c25-891da9aee43c" (UID: "d9f74a60-e9aa-4366-9c25-891da9aee43c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.733994 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.733689    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-data\") pod \"d9f74a60-e9aa-4366-9c25-891da9aee43c\" (UID: \"d9f74a60-e9aa-4366-9c25-891da9aee43c\") "
Oct 30 22:03:08.733994 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.733695    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-workload-socket" pod "cd9299f1-9812-4f6c-8851-6957b8b3ab64" (UID: "cd9299f1-9812-4f6c-8851-6957b8b3ab64"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.734572 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.734544    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-data" pod "cd9299f1-9812-4f6c-8851-6957b8b3ab64" (UID: "cd9299f1-9812-4f6c-8851-6957b8b3ab64"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.734902 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.734875    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd9299f1-9812-4f6c-8851-6957b8b3ab64-kube-api-access-s47tj" pod "cd9299f1-9812-4f6c-8851-6957b8b3ab64" (UID: "cd9299f1-9812-4f6c-8851-6957b8b3ab64"). InnerVolumeSpecName "kube-api-access-s47tj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:08.734977 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.734949    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-envoy" pod "cd9299f1-9812-4f6c-8851-6957b8b3ab64" (UID: "cd9299f1-9812-4f6c-8851-6957b8b3ab64"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.735402 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.735377    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-envoy" pod "d9f74a60-e9aa-4366-9c25-891da9aee43c" (UID: "d9f74a60-e9aa-4366-9c25-891da9aee43c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.735492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.735447    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istiod-ca-cert" pod "5568d46b-ed6b-4c97-99e0-8a0de7cc5258" (UID: "5568d46b-ed6b-4c97-99e0-8a0de7cc5258"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:08.735555 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.735522    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-podinfo" pod "5568d46b-ed6b-4c97-99e0-8a0de7cc5258" (UID: "5568d46b-ed6b-4c97-99e0-8a0de7cc5258"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:03:08.735726 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.735704    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-podinfo" pod "d9f74a60-e9aa-4366-9c25-891da9aee43c" (UID: "d9f74a60-e9aa-4366-9c25-891da9aee43c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:03:08.735784 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.735700    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-token" pod "5568d46b-ed6b-4c97-99e0-8a0de7cc5258" (UID: "5568d46b-ed6b-4c97-99e0-8a0de7cc5258"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:08.735834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.735777    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-workload-certs" pod "5568d46b-ed6b-4c97-99e0-8a0de7cc5258" (UID: "5568d46b-ed6b-4c97-99e0-8a0de7cc5258"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.735887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.735839    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istiod-ca-cert" pod "cd9299f1-9812-4f6c-8851-6957b8b3ab64" (UID: "cd9299f1-9812-4f6c-8851-6957b8b3ab64"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:08.736006 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.735978    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-data" pod "d9f74a60-e9aa-4366-9c25-891da9aee43c" (UID: "d9f74a60-e9aa-4366-9c25-891da9aee43c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.736206 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.736184    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9f74a60-e9aa-4366-9c25-891da9aee43c-istiod-ca-cert" pod "d9f74a60-e9aa-4366-9c25-891da9aee43c" (UID: "d9f74a60-e9aa-4366-9c25-891da9aee43c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:08.737128 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.737103    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-podinfo" pod "cd9299f1-9812-4f6c-8851-6957b8b3ab64" (UID: "cd9299f1-9812-4f6c-8851-6957b8b3ab64"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:03:08.737764 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.737745    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-kube-api-access-68fz6" pod "5568d46b-ed6b-4c97-99e0-8a0de7cc5258" (UID: "5568d46b-ed6b-4c97-99e0-8a0de7cc5258"). InnerVolumeSpecName "kube-api-access-68fz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:08.737932 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.737899    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-envoy" pod "5568d46b-ed6b-4c97-99e0-8a0de7cc5258" (UID: "5568d46b-ed6b-4c97-99e0-8a0de7cc5258"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:08.740685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740660    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.740792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740689    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.740792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740705    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.740792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740726    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.740792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740739    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.740792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740753    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.740792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740767    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.740792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740784    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740798    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740812    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740825    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740853    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740868    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740881    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740894    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740919    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740933    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740947    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740965    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.740978    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.741002    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.741015    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2s62\" (UniqueName: \"kubernetes.io/projected/c145c160-382f-4151-bfd1-cab6a9e4dc98-kube-api-access-l2s62\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.741036    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.741049    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68fz6\" (UniqueName: \"kubernetes.io/projected/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-kube-api-access-68fz6\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.741063    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s47tj\" (UniqueName: \"kubernetes.io/projected/cd9299f1-9812-4f6c-8851-6957b8b3ab64-kube-api-access-s47tj\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.741116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.741122    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c145c160-382f-4151-bfd1-cab6a9e4dc98-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.742224 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.741142    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.742224 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.741159    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9299f1-9812-4f6c-8851-6957b8b3ab64-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.742224 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.741181    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.742224 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.741194    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c145c160-382f-4151-bfd1-cab6a9e4dc98-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.742224 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.741300    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5568d46b-ed6b-4c97-99e0-8a0de7cc5258-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.742224 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.741316    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9f74a60-e9aa-4366-9c25-891da9aee43c-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.742224 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.741331    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.753973 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.753940    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9f74a60-e9aa-4366-9c25-891da9aee43c-kube-api-access-w7gr8" pod "d9f74a60-e9aa-4366-9c25-891da9aee43c" (UID: "d9f74a60-e9aa-4366-9c25-891da9aee43c"). InnerVolumeSpecName "kube-api-access-w7gr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:08.754115 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.753990    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-token" pod "d9f74a60-e9aa-4366-9c25-891da9aee43c" (UID: "d9f74a60-e9aa-4366-9c25-891da9aee43c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:08.755200 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.755170    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-token" pod "cd9299f1-9812-4f6c-8851-6957b8b3ab64" (UID: "cd9299f1-9812-4f6c-8851-6957b8b3ab64"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:08.843752 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.843715    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9f74a60-e9aa-4366-9c25-891da9aee43c-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.843752 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.843746    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd9299f1-9812-4f6c-8851-6957b8b3ab64-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:08.843752 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:08.843758    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7gr8\" (UniqueName: \"kubernetes.io/projected/d9f74a60-e9aa-4366-9c25-891da9aee43c-kube-api-access-w7gr8\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:09.021530 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.021412    5586 generic.go:356] "Generic (PLEG): container finished" podID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerID="de84582f74c1b9b6cf0e326d08d3fa53c47e6e83113534ca0ec59e30e2d31fb0" exitCode=0
Oct 30 22:03:09.021530 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.021488    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"
Oct 30 22:03:09.021530 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.021496    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" event={"ID":"c145c160-382f-4151-bfd1-cab6a9e4dc98","Type":"ContainerDied","Data":"de84582f74c1b9b6cf0e326d08d3fa53c47e6e83113534ca0ec59e30e2d31fb0"}
Oct 30 22:03:09.022204 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.021619    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2" event={"ID":"c145c160-382f-4151-bfd1-cab6a9e4dc98","Type":"ContainerDied","Data":"4a5061d2b088ae4420e12f46049e0ea97a66af624427485ed58892adc630e159"}
Oct 30 22:03:09.022204 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.021671    5586 scope.go:122] "RemoveContainer" containerID="de84582f74c1b9b6cf0e326d08d3fa53c47e6e83113534ca0ec59e30e2d31fb0"
Oct 30 22:03:09.025315 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.025284    5586 generic.go:356] "Generic (PLEG): container finished" podID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerID="49626e30c184a07ec9f242c441f2cad15c993f1b4e120cf51fafa38ac77ad15c" exitCode=0
Oct 30 22:03:09.025469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.025403    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"
Oct 30 22:03:09.029451 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.029418    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd" event={"ID":"cd9299f1-9812-4f6c-8851-6957b8b3ab64","Type":"ContainerDied","Data":"49626e30c184a07ec9f242c441f2cad15c993f1b4e120cf51fafa38ac77ad15c"}
Oct 30 22:03:09.029646 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.029482    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd" event={"ID":"cd9299f1-9812-4f6c-8851-6957b8b3ab64","Type":"ContainerDied","Data":"7bcf250a2a0b73cc6db0eee96e257346f23e6c246ef17fbd03028fcb9c8785c3"}
Oct 30 22:03:09.035855 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.035576    5586 generic.go:356] "Generic (PLEG): container finished" podID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" containerID="c13342f0489dbdc044381f97127790e3f503c4c978f4a7c77d58a31d1247eff8" exitCode=0
Oct 30 22:03:09.035855 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.035645    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"
Oct 30 22:03:09.035855 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.035756    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" event={"ID":"5568d46b-ed6b-4c97-99e0-8a0de7cc5258","Type":"ContainerDied","Data":"c13342f0489dbdc044381f97127790e3f503c4c978f4a7c77d58a31d1247eff8"}
Oct 30 22:03:09.035855 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.035799    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8" event={"ID":"5568d46b-ed6b-4c97-99e0-8a0de7cc5258","Type":"ContainerDied","Data":"33b13ede262c3a0dd11167439c64cfc3d71b607c071c630294727332f4266288"}
Oct 30 22:03:09.037537 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.037511    5586 generic.go:356] "Generic (PLEG): container finished" podID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerID="29b68d69c49d91bf30607ad98540ddb3439012f394ada62d82db52543455229b" exitCode=0
Oct 30 22:03:09.040437 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.037595    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" event={"ID":"d9f74a60-e9aa-4366-9c25-891da9aee43c","Type":"ContainerDied","Data":"29b68d69c49d91bf30607ad98540ddb3439012f394ada62d82db52543455229b"}
Oct 30 22:03:09.040437 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.037632    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx" event={"ID":"d9f74a60-e9aa-4366-9c25-891da9aee43c","Type":"ContainerDied","Data":"18e5d1dfbae5252d39203fb4f7df0a0946ba1f0ec68cf2554b8f6294cc33dc6e"}
Oct 30 22:03:09.040437 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.037641    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"
Oct 30 22:03:09.042442 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.042412    5586 scope.go:122] "RemoveContainer" containerID="de84582f74c1b9b6cf0e326d08d3fa53c47e6e83113534ca0ec59e30e2d31fb0"
Oct 30 22:03:09.045423 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:09.045388    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de84582f74c1b9b6cf0e326d08d3fa53c47e6e83113534ca0ec59e30e2d31fb0\": container with ID starting with de84582f74c1b9b6cf0e326d08d3fa53c47e6e83113534ca0ec59e30e2d31fb0 not found: ID does not exist" containerID="de84582f74c1b9b6cf0e326d08d3fa53c47e6e83113534ca0ec59e30e2d31fb0"
Oct 30 22:03:09.045556 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.045434    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de84582f74c1b9b6cf0e326d08d3fa53c47e6e83113534ca0ec59e30e2d31fb0"} err="failed to get container status \"de84582f74c1b9b6cf0e326d08d3fa53c47e6e83113534ca0ec59e30e2d31fb0\": rpc error: code = NotFound desc = could not find container \"de84582f74c1b9b6cf0e326d08d3fa53c47e6e83113534ca0ec59e30e2d31fb0\": container with ID starting with de84582f74c1b9b6cf0e326d08d3fa53c47e6e83113534ca0ec59e30e2d31fb0 not found: ID does not exist"
Oct 30 22:03:09.045556 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.045464    5586 scope.go:122] "RemoveContainer" containerID="49626e30c184a07ec9f242c441f2cad15c993f1b4e120cf51fafa38ac77ad15c"
Oct 30 22:03:09.062350 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.062326    5586 scope.go:122] "RemoveContainer" containerID="803b026b4bc0d8aa4d34faa0bef1c98e42c98b35add07399fd76f20e4bf51ab3"
Oct 30 22:03:09.074733 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.074703    5586 scope.go:122] "RemoveContainer" containerID="49626e30c184a07ec9f242c441f2cad15c993f1b4e120cf51fafa38ac77ad15c"
Oct 30 22:03:09.075130 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:09.075107    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49626e30c184a07ec9f242c441f2cad15c993f1b4e120cf51fafa38ac77ad15c\": container with ID starting with 49626e30c184a07ec9f242c441f2cad15c993f1b4e120cf51fafa38ac77ad15c not found: ID does not exist" containerID="49626e30c184a07ec9f242c441f2cad15c993f1b4e120cf51fafa38ac77ad15c"
Oct 30 22:03:09.075278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.075142    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49626e30c184a07ec9f242c441f2cad15c993f1b4e120cf51fafa38ac77ad15c"} err="failed to get container status \"49626e30c184a07ec9f242c441f2cad15c993f1b4e120cf51fafa38ac77ad15c\": rpc error: code = NotFound desc = could not find container \"49626e30c184a07ec9f242c441f2cad15c993f1b4e120cf51fafa38ac77ad15c\": container with ID starting with 49626e30c184a07ec9f242c441f2cad15c993f1b4e120cf51fafa38ac77ad15c not found: ID does not exist"
Oct 30 22:03:09.075278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.075165    5586 scope.go:122] "RemoveContainer" containerID="803b026b4bc0d8aa4d34faa0bef1c98e42c98b35add07399fd76f20e4bf51ab3"
Oct 30 22:03:09.075557 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:09.075525    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"803b026b4bc0d8aa4d34faa0bef1c98e42c98b35add07399fd76f20e4bf51ab3\": container with ID starting with 803b026b4bc0d8aa4d34faa0bef1c98e42c98b35add07399fd76f20e4bf51ab3 not found: ID does not exist" containerID="803b026b4bc0d8aa4d34faa0bef1c98e42c98b35add07399fd76f20e4bf51ab3"
Oct 30 22:03:09.075626 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.075565    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"803b026b4bc0d8aa4d34faa0bef1c98e42c98b35add07399fd76f20e4bf51ab3"} err="failed to get container status \"803b026b4bc0d8aa4d34faa0bef1c98e42c98b35add07399fd76f20e4bf51ab3\": rpc error: code = NotFound desc = could not find container \"803b026b4bc0d8aa4d34faa0bef1c98e42c98b35add07399fd76f20e4bf51ab3\": container with ID starting with 803b026b4bc0d8aa4d34faa0bef1c98e42c98b35add07399fd76f20e4bf51ab3 not found: ID does not exist"
Oct 30 22:03:09.075626 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.075586    5586 scope.go:122] "RemoveContainer" containerID="c13342f0489dbdc044381f97127790e3f503c4c978f4a7c77d58a31d1247eff8"
Oct 30 22:03:09.120720 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.120674    5586 scope.go:122] "RemoveContainer" containerID="c13342f0489dbdc044381f97127790e3f503c4c978f4a7c77d58a31d1247eff8"
Oct 30 22:03:09.121043 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:09.121021    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c13342f0489dbdc044381f97127790e3f503c4c978f4a7c77d58a31d1247eff8\": container with ID starting with c13342f0489dbdc044381f97127790e3f503c4c978f4a7c77d58a31d1247eff8 not found: ID does not exist" containerID="c13342f0489dbdc044381f97127790e3f503c4c978f4a7c77d58a31d1247eff8"
Oct 30 22:03:09.121120 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.121057    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c13342f0489dbdc044381f97127790e3f503c4c978f4a7c77d58a31d1247eff8"} err="failed to get container status \"c13342f0489dbdc044381f97127790e3f503c4c978f4a7c77d58a31d1247eff8\": rpc error: code = NotFound desc = could not find container \"c13342f0489dbdc044381f97127790e3f503c4c978f4a7c77d58a31d1247eff8\": container with ID starting with c13342f0489dbdc044381f97127790e3f503c4c978f4a7c77d58a31d1247eff8 not found: ID does not exist"
Oct 30 22:03:09.121120 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.121085    5586 scope.go:122] "RemoveContainer" containerID="29b68d69c49d91bf30607ad98540ddb3439012f394ada62d82db52543455229b"
Oct 30 22:03:09.140511 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.138116    5586 scope.go:122] "RemoveContainer" containerID="29b68d69c49d91bf30607ad98540ddb3439012f394ada62d82db52543455229b"
Oct 30 22:03:09.140511 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:09.138569    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29b68d69c49d91bf30607ad98540ddb3439012f394ada62d82db52543455229b\": container with ID starting with 29b68d69c49d91bf30607ad98540ddb3439012f394ada62d82db52543455229b not found: ID does not exist" containerID="29b68d69c49d91bf30607ad98540ddb3439012f394ada62d82db52543455229b"
Oct 30 22:03:09.140511 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.138626    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29b68d69c49d91bf30607ad98540ddb3439012f394ada62d82db52543455229b"} err="failed to get container status \"29b68d69c49d91bf30607ad98540ddb3439012f394ada62d82db52543455229b\": rpc error: code = NotFound desc = could not find container \"29b68d69c49d91bf30607ad98540ddb3439012f394ada62d82db52543455229b\": container with ID starting with 29b68d69c49d91bf30607ad98540ddb3439012f394ada62d82db52543455229b not found: ID does not exist"
Oct 30 22:03:09.263033 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.262995    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"]
Oct 30 22:03:09.267460 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.267427    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-bxblx"]
Oct 30 22:03:09.371640 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.371548    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"]
Oct 30 22:03:09.375158 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.375136    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-g4brd"]
Oct 30 22:03:09.449484 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.449452    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" path="/var/lib/kubelet/pods/cd9299f1-9812-4f6c-8851-6957b8b3ab64/volumes"
Oct 30 22:03:09.450085 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.450068    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" path="/var/lib/kubelet/pods/d9f74a60-e9aa-4366-9c25-891da9aee43c/volumes"
Oct 30 22:03:09.482721 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.481907    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"]
Oct 30 22:03:09.485965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.485944    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-m8zc2"]
Oct 30 22:03:09.595391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.595359    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"]
Oct 30 22:03:09.599996 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:09.599969    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-c3e2d8-openshift-default-5b5d87968c-rk8g8"]
Oct 30 22:03:10.172839 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:10.172804    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:11.048266 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.048154    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:11.260138 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.260104    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:11.449475 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.449442    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" path="/var/lib/kubelet/pods/5568d46b-ed6b-4c97-99e0-8a0de7cc5258/volumes"
Oct 30 22:03:11.450017 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.450001    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" path="/var/lib/kubelet/pods/c145c160-382f-4151-bfd1-cab6a9e4dc98/volumes"
Oct 30 22:03:11.454609 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.454584    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"]
Oct 30 22:03:11.455326 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.455311    5586 cpu_manager.go:414] "containerMap: removing container" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy"
Oct 30 22:03:11.455393 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.455327    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy"
Oct 30 22:03:11.455393 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.455340    5586 cpu_manager.go:414] "containerMap: removing container" podUID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" containerName="istio-proxy"
Oct 30 22:03:11.455393 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.455345    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" containerName="istio-proxy"
Oct 30 22:03:11.455393 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.455378    5586 cpu_manager.go:414] "containerMap: removing container" podUID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerName="istio-proxy"
Oct 30 22:03:11.455393 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.455383    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerName="istio-proxy"
Oct 30 22:03:11.455393 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.455392    5586 cpu_manager.go:414] "containerMap: removing container" podUID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerName="istio-proxy"
Oct 30 22:03:11.455574 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.455397    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerName="istio-proxy"
Oct 30 22:03:11.455574 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.455405    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy"
Oct 30 22:03:11.455574 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.455410    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy"
Oct 30 22:03:11.455574 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.455499    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerName="istio-proxy"
Oct 30 22:03:11.455574 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.455508    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9f74a60-e9aa-4366-9c25-891da9aee43c" containerName="istio-proxy"
Oct 30 22:03:11.455574 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.455519    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="c145c160-382f-4151-bfd1-cab6a9e4dc98" containerName="istio-proxy"
Oct 30 22:03:11.455574 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.455525    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="5568d46b-ed6b-4c97-99e0-8a0de7cc5258" containerName="istio-proxy"
Oct 30 22:03:11.460992 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.460971    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"
Oct 30 22:03:11.536552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.536503    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b73ddad4-9be5-4fa2-9691-013116c525f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd\" (UID: \"b73ddad4-9be5-4fa2-9691-013116c525f7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"
Oct 30 22:03:11.536733 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.536652    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n84r\" (UniqueName: \"kubernetes.io/projected/b73ddad4-9be5-4fa2-9691-013116c525f7-kube-api-access-4n84r\") pod \"payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd\" (UID: \"b73ddad4-9be5-4fa2-9691-013116c525f7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"
Oct 30 22:03:11.553662 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.553618    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"]
Oct 30 22:03:11.579246 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.579194    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-ea3cb2-dockercfg-lgppz\""
Oct 30 22:03:11.580408 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.580361    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-ea3cb2\""
Oct 30 22:03:11.601633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.601601    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"]
Oct 30 22:03:11.602501 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.602478    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd9299f1-9812-4f6c-8851-6957b8b3ab64" containerName="istio-proxy"
Oct 30 22:03:11.606694 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.606665    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"
Oct 30 22:03:11.637931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.637894    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n84r\" (UniqueName: \"kubernetes.io/projected/b73ddad4-9be5-4fa2-9691-013116c525f7-kube-api-access-4n84r\") pod \"payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd\" (UID: \"b73ddad4-9be5-4fa2-9691-013116c525f7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"
Oct 30 22:03:11.638182 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.638160    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b73ddad4-9be5-4fa2-9691-013116c525f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd\" (UID: \"b73ddad4-9be5-4fa2-9691-013116c525f7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"
Oct 30 22:03:11.638282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.638213    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfeb8098-593b-437f-acb7-74348724dea2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l\" (UID: \"dfeb8098-593b-437f-acb7-74348724dea2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"
Oct 30 22:03:11.638353 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.638279    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jssbx\" (UniqueName: \"kubernetes.io/projected/dfeb8098-593b-437f-acb7-74348724dea2-kube-api-access-jssbx\") pod \"payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l\" (UID: \"dfeb8098-593b-437f-acb7-74348724dea2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"
Oct 30 22:03:11.639282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.639254    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b73ddad4-9be5-4fa2-9691-013116c525f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd\" (UID: \"b73ddad4-9be5-4fa2-9691-013116c525f7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"
Oct 30 22:03:11.661357 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.661323    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"]
Oct 30 22:03:11.698391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.698351    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n84r\" (UniqueName: \"kubernetes.io/projected/b73ddad4-9be5-4fa2-9691-013116c525f7-kube-api-access-4n84r\") pod \"payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd\" (UID: \"b73ddad4-9be5-4fa2-9691-013116c525f7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"
Oct 30 22:03:11.747328 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.740132    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfeb8098-593b-437f-acb7-74348724dea2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l\" (UID: \"dfeb8098-593b-437f-acb7-74348724dea2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"
Oct 30 22:03:11.747328 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.740218    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jssbx\" (UniqueName: \"kubernetes.io/projected/dfeb8098-593b-437f-acb7-74348724dea2-kube-api-access-jssbx\") pod \"payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l\" (UID: \"dfeb8098-593b-437f-acb7-74348724dea2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"
Oct 30 22:03:11.747328 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.741722    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfeb8098-593b-437f-acb7-74348724dea2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l\" (UID: \"dfeb8098-593b-437f-acb7-74348724dea2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"
Oct 30 22:03:11.777138 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.777084    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"
Oct 30 22:03:11.797984 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.797952    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jssbx\" (UniqueName: \"kubernetes.io/projected/dfeb8098-593b-437f-acb7-74348724dea2-kube-api-access-jssbx\") pod \"payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l\" (UID: \"dfeb8098-593b-437f-acb7-74348724dea2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"
Oct 30 22:03:11.931832 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:11.931795    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"
Oct 30 22:03:12.189024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:12.188993    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"]
Oct 30 22:03:12.189521 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:03:12.189494    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73ddad4_9be5_4fa2_9691_013116c525f7.slice/crio-d14a7cc01388f8640c273da5f693cf20cb0130945788727d61ce7343761c8fe3 WatchSource:0}: Error finding container d14a7cc01388f8640c273da5f693cf20cb0130945788727d61ce7343761c8fe3: Status 404 returned error can't find the container with id d14a7cc01388f8640c273da5f693cf20cb0130945788727d61ce7343761c8fe3
Oct 30 22:03:12.317815 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:12.317789    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"]
Oct 30 22:03:12.320355 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:03:12.320321    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfeb8098_593b_437f_acb7_74348724dea2.slice/crio-c12dfeaf708ecaf152ded46bbc1c4212796d2c180975e8cba86f860157275e41 WatchSource:0}: Error finding container c12dfeaf708ecaf152ded46bbc1c4212796d2c180975e8cba86f860157275e41: Status 404 returned error can't find the container with id c12dfeaf708ecaf152ded46bbc1c4212796d2c180975e8cba86f860157275e41
Oct 30 22:03:12.443679 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:12.443638    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:13.078973 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:13.078936    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd" event={"ID":"b73ddad4-9be5-4fa2-9691-013116c525f7","Type":"ContainerStarted","Data":"8c4dd6c4878de9e9635af49ea4f2eae9073c81c1a664df0765b742be1c99620f"}
Oct 30 22:03:13.079151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:13.078997    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd" event={"ID":"b73ddad4-9be5-4fa2-9691-013116c525f7","Type":"ContainerStarted","Data":"d14a7cc01388f8640c273da5f693cf20cb0130945788727d61ce7343761c8fe3"}
Oct 30 22:03:13.079151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:13.079028    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"
Oct 30 22:03:13.080552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:13.080521    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l" event={"ID":"dfeb8098-593b-437f-acb7-74348724dea2","Type":"ContainerStarted","Data":"16cac7c0e3bd8f1d511e48b1f17946ed62456efd88278ff89f5e6d28de626bd2"}
Oct 30 22:03:13.080681 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:13.080566    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l" event={"ID":"dfeb8098-593b-437f-acb7-74348724dea2","Type":"ContainerStarted","Data":"c12dfeaf708ecaf152ded46bbc1c4212796d2c180975e8cba86f860157275e41"}
Oct 30 22:03:13.080681 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:13.080602    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"
Oct 30 22:03:13.196432 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:13.196370    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd" podStartSLOduration=2.196351513 podStartE2EDuration="2.196351513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:13.193348123 +0000 UTC m=+1735.865252058" watchObservedRunningTime="2026-10-30 22:03:13.196351513 +0000 UTC m=+1735.868255449"
Oct 30 22:03:13.198345 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:13.198312    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:13.311097 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:13.311038    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l" podStartSLOduration=2.311023156 podStartE2EDuration="2.311023156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:13.3100089 +0000 UTC m=+1735.981912835" watchObservedRunningTime="2026-10-30 22:03:13.311023156 +0000 UTC m=+1735.982927090"
Oct 30 22:03:13.384254 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:13.384161    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:13.564987 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:13.564955    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:16.553446 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:16.553410    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:16.736940 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:16.736900    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:18.146114 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:18.146080    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"]
Oct 30 22:03:18.146659 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:18.146323    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" containerID="cri-o://029c8bac358e1267ce2cc358cf70a088509f3eeb3d58e079761a769b5648d89c" gracePeriod=30
Oct 30 22:03:18.186473 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:18.186437    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:18.202219 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:18.202182    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"]
Oct 30 22:03:18.202907 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:18.202854    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy" containerID="cri-o://2dd946dde2ce3cd9b36290df05ea6bf79b6994af1421bbf5b7c9afaf6cba5ff5" gracePeriod=30
Oct 30 22:03:18.212397 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:18.212359    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"]
Oct 30 22:03:18.212716 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:18.212668    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5" podUID="a5989307-0754-4796-a7f4-79a20accfbd5" containerName="istio-proxy" containerID="cri-o://ba6830afd8567063ccff765c08fd4bbc3cc2e495eb8f2f8af91d5bc3051a5f41" gracePeriod=30
Oct 30 22:03:18.214928 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:18.214899    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"]
Oct 30 22:03:18.215136 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:18.215112    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" containerID="cri-o://fccc81d858c2daaf1cd4641d0d91067b25781a41ff3a6a42686a94f678efd869" gracePeriod=30
Oct 30 22:03:18.368372 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:18.368332    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:03:18.884311 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:18.884273    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:03:18.884531 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:18.884335    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:03:19.281417 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:19.280165    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:03:19.281417 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:19.280260    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:03:19.745485 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:19.745450    5586 scope.go:122] "RemoveContainer" containerID="692cffdd6b69ac6449870e330ea934bfdf0b91a1d935d7cbbc58ddfa38427582"
Oct 30 22:03:19.756500 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:19.756475    5586 scope.go:122] "RemoveContainer" containerID="dfb262670e3e32ddeaba9413a49d1c06009f48af6c3bf22568764b1db95e4109"
Oct 30 22:03:19.766145 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:19.766105    5586 scope.go:122] "RemoveContainer" containerID="fdc1c48895e6b8d6dd9928e8fc970813444244af128f983963d00416c0c67e9f"
Oct 30 22:03:19.775501 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:19.775478    5586 scope.go:122] "RemoveContainer" containerID="2de51ed0df230931e42cb701379e4adc86a0ef0d78971eb69db2179440dab486"
Oct 30 22:03:19.785024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:19.785004    5586 scope.go:122] "RemoveContainer" containerID="d3c71b55f51be831ccd90ab32288ba5bd03e22efb5cd127107bab0c921057496"
Oct 30 22:03:21.411057 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:21.411020    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:03:21.411492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:21.411080    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5" podUID="a5989307-0754-4796-a7f4-79a20accfbd5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:03:22.720835 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.720793    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"]
Oct 30 22:03:22.728107 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.728081    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:22.783566 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.783531    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"]
Oct 30 22:03:22.817133 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.817085    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a469237-ca99-4126-938b-bcbee6d9835f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:22.817341 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.817317    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:22.817407 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.817359    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:22.817472 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.817454    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mth5p\" (UniqueName: \"kubernetes.io/projected/5a469237-ca99-4126-938b-bcbee6d9835f-kube-api-access-mth5p\") pod \"maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:22.827549 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.827509    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-99f69c-serving-cert\""
Oct 30 22:03:22.919447 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.919410    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a469237-ca99-4126-938b-bcbee6d9835f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:22.919623 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.919525    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:22.919623 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.919546    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:22.919623 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.919577    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mth5p\" (UniqueName: \"kubernetes.io/projected/5a469237-ca99-4126-938b-bcbee6d9835f-kube-api-access-mth5p\") pod \"maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:22.919743 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:22.919678    5586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-serving-cert: secret "maas-api-e2e-shared-a-w0-99f69c-serving-cert" not found
Oct 30 22:03:22.919801 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:22.919788    5586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-maas-api-tls podName:5a469237-ca99-4126-938b-bcbee6d9835f nodeName:}" failed. No retries permitted until 2026-10-30 22:03:23.419763901 +0000 UTC m=+1746.091667838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-maas-api-tls") pod "maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp" (UID: "5a469237-ca99-4126-938b-bcbee6d9835f") : secret "maas-api-e2e-shared-a-w0-99f69c-serving-cert" not found
Oct 30 22:03:22.920263 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.920224    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a469237-ca99-4126-938b-bcbee6d9835f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:22.922065 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.922046    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:22.924565 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.924534    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:22.982502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:22.982423    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mth5p\" (UniqueName: \"kubernetes.io/projected/5a469237-ca99-4126-938b-bcbee6d9835f-kube-api-access-mth5p\") pod \"maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:23.098076 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.098032    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:23.258225 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.258190    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:23.431474 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.431437    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:23.434025 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.433999    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:23.507021 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.506459    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 22:03:23.545272 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.543020    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.670780    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-credential-socket\") pod \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.670838    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-workload-certs\") pod \"a2297dba-5385-44d4-b0ae-1e470b525fc9\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.670878    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-podinfo\") pod \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.670943    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-podinfo\") pod \"a2297dba-5385-44d4-b0ae-1e470b525fc9\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.670977    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2297dba-5385-44d4-b0ae-1e470b525fc9-kube-api-access-5crgr\" (UniqueName: \"kubernetes.io/projected/a2297dba-5385-44d4-b0ae-1e470b525fc9-kube-api-access-5crgr\") pod \"a2297dba-5385-44d4-b0ae-1e470b525fc9\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.671011    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-workload-socket\") pod \"a2297dba-5385-44d4-b0ae-1e470b525fc9\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.671086    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istiod-ca-cert\") pod \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.671155    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-token\" (UniqueName: \"kubernetes.io/projected/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-token\") pod \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.671208    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2297dba-5385-44d4-b0ae-1e470b525fc9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2297dba-5385-44d4-b0ae-1e470b525fc9-istiod-ca-cert\") pod \"a2297dba-5385-44d4-b0ae-1e470b525fc9\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.671276    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-workload-certs\") pod \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.671377    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-credential-socket\") pod \"a2297dba-5385-44d4-b0ae-1e470b525fc9\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.671407    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-data\") pod \"a2297dba-5385-44d4-b0ae-1e470b525fc9\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.671443    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-workload-socket\") pod \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.671510    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-envoy\") pod \"a2297dba-5385-44d4-b0ae-1e470b525fc9\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.671584    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-data\") pod \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.671654    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-envoy\") pod \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.671694    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-token\" (UniqueName: \"kubernetes.io/projected/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-token\") pod \"a2297dba-5385-44d4-b0ae-1e470b525fc9\" (UID: \"a2297dba-5385-44d4-b0ae-1e470b525fc9\") "
Oct 30 22:03:23.672698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.671795    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-kube-api-access-9gvbg\" (UniqueName: \"kubernetes.io/projected/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-kube-api-access-9gvbg\") pod \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\" (UID: \"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd\") "
Oct 30 22:03:23.677609 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.676603    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:23.677609 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.677337    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-kube-api-access-9gvbg" pod "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" (UID: "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd"). InnerVolumeSpecName "kube-api-access-9gvbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:23.679010 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.678649    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2297dba-5385-44d4-b0ae-1e470b525fc9-istiod-ca-cert" pod "a2297dba-5385-44d4-b0ae-1e470b525fc9" (UID: "a2297dba-5385-44d4-b0ae-1e470b525fc9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:23.679591 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.679193    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-workload-certs" pod "a2297dba-5385-44d4-b0ae-1e470b525fc9" (UID: "a2297dba-5385-44d4-b0ae-1e470b525fc9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.688302 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.681497    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-data" pod "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" (UID: "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.689289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.689213    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-workload-socket" pod "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" (UID: "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.689539 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.689520    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-workload-socket" pod "a2297dba-5385-44d4-b0ae-1e470b525fc9" (UID: "a2297dba-5385-44d4-b0ae-1e470b525fc9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.691038 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.691016    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istiod-ca-cert" pod "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" (UID: "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:23.695133 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.694869    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-podinfo" pod "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" (UID: "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:03:23.695360 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.695323    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-workload-certs" pod "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" (UID: "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.695540 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.695515    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-credential-socket" pod "a2297dba-5385-44d4-b0ae-1e470b525fc9" (UID: "a2297dba-5385-44d4-b0ae-1e470b525fc9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.695617 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.695527    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-credential-socket" pod "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" (UID: "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.695741 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.695718    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-data" pod "a2297dba-5385-44d4-b0ae-1e470b525fc9" (UID: "a2297dba-5385-44d4-b0ae-1e470b525fc9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.696557 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.696529    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-podinfo" pod "a2297dba-5385-44d4-b0ae-1e470b525fc9" (UID: "a2297dba-5385-44d4-b0ae-1e470b525fc9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:03:23.697380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.697359    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2297dba-5385-44d4-b0ae-1e470b525fc9-kube-api-access-5crgr" pod "a2297dba-5385-44d4-b0ae-1e470b525fc9" (UID: "a2297dba-5385-44d4-b0ae-1e470b525fc9"). InnerVolumeSpecName "kube-api-access-5crgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:23.699041 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.699015    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-token" pod "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" (UID: "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:23.703839 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.703801    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-envoy" pod "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" (UID: "77c5d7c9-2c19-4df8-a2ff-2e159881e1fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.704122 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.704085    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-envoy" pod "a2297dba-5385-44d4-b0ae-1e470b525fc9" (UID: "a2297dba-5385-44d4-b0ae-1e470b525fc9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.706631 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.706599    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-token" pod "a2297dba-5385-44d4-b0ae-1e470b525fc9" (UID: "a2297dba-5385-44d4-b0ae-1e470b525fc9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:23.724405 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.724377    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 22:03:23.735655 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.735525    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 22:03:23.780212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.778209    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.780212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.778283    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.780212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.778302    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gvbg\" (UniqueName: \"kubernetes.io/projected/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-kube-api-access-9gvbg\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.780212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.778319    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.780212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.778335    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.780212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.778350    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.780212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.778363    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.780212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.778377    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5crgr\" (UniqueName: \"kubernetes.io/projected/a2297dba-5385-44d4-b0ae-1e470b525fc9-kube-api-access-5crgr\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.781017 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.780993    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.781075 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.781029    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.781075 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.781044    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.781075 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.781057    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2297dba-5385-44d4-b0ae-1e470b525fc9-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.781075 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.781072    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.781212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.781086    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.781212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.781100    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.781212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.781114    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.781212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.781132    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2297dba-5385-44d4-b0ae-1e470b525fc9-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.781212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.781145    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.882190 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.882148    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc3e1a16-31d6-4f92-b201-0672d2b02946-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc3e1a16-31d6-4f92-b201-0672d2b02946-istiod-ca-cert\") pod \"fc3e1a16-31d6-4f92-b201-0672d2b02946\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") "
Oct 30 22:03:23.882378 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.882298    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-token\" (UniqueName: \"kubernetes.io/projected/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-token\") pod \"fc3e1a16-31d6-4f92-b201-0672d2b02946\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") "
Oct 30 22:03:23.882421 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.882375    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-workload-certs\") pod \"a5989307-0754-4796-a7f4-79a20accfbd5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") "
Oct 30 22:03:23.882482 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.882455    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-credential-socket\") pod \"fc3e1a16-31d6-4f92-b201-0672d2b02946\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") "
Oct 30 22:03:23.882561 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.882531    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a5989307-0754-4796-a7f4-79a20accfbd5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5989307-0754-4796-a7f4-79a20accfbd5-istio-podinfo\") pod \"a5989307-0754-4796-a7f4-79a20accfbd5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") "
Oct 30 22:03:23.882622 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.882571    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-data\") pod \"fc3e1a16-31d6-4f92-b201-0672d2b02946\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") "
Oct 30 22:03:23.882687 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.882648    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-credential-socket\") pod \"a5989307-0754-4796-a7f4-79a20accfbd5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") "
Oct 30 22:03:23.882742 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.882683    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc3e1a16-31d6-4f92-b201-0672d2b02946-kube-api-access-7bw7n\" (UniqueName: \"kubernetes.io/projected/fc3e1a16-31d6-4f92-b201-0672d2b02946-kube-api-access-7bw7n\") pod \"fc3e1a16-31d6-4f92-b201-0672d2b02946\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") "
Oct 30 22:03:23.882805 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.882778    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-envoy\") pod \"fc3e1a16-31d6-4f92-b201-0672d2b02946\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") "
Oct 30 22:03:23.882859 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.882848    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-workload-socket\") pod \"a5989307-0754-4796-a7f4-79a20accfbd5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") "
Oct 30 22:03:23.882914 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.882902    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-workload-socket\") pod \"fc3e1a16-31d6-4f92-b201-0672d2b02946\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") "
Oct 30 22:03:23.882965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.882951    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-istio-envoy\") pod \"a5989307-0754-4796-a7f4-79a20accfbd5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") "
Oct 30 22:03:23.883030 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.882982    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5989307-0754-4796-a7f4-79a20accfbd5-istio-token\" (UniqueName: \"kubernetes.io/projected/a5989307-0754-4796-a7f4-79a20accfbd5-istio-token\") pod \"a5989307-0754-4796-a7f4-79a20accfbd5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") "
Oct 30 22:03:23.883030 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.883020    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-workload-certs\") pod \"fc3e1a16-31d6-4f92-b201-0672d2b02946\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") "
Oct 30 22:03:23.883143 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.883059    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-istio-data\") pod \"a5989307-0754-4796-a7f4-79a20accfbd5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") "
Oct 30 22:03:23.883143 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.883112    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5989307-0754-4796-a7f4-79a20accfbd5-kube-api-access-7bq68\" (UniqueName: \"kubernetes.io/projected/a5989307-0754-4796-a7f4-79a20accfbd5-kube-api-access-7bq68\") pod \"a5989307-0754-4796-a7f4-79a20accfbd5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") "
Oct 30 22:03:23.883261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.883162    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-credential-socket" pod "fc3e1a16-31d6-4f92-b201-0672d2b02946" (UID: "fc3e1a16-31d6-4f92-b201-0672d2b02946"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.883261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.883171    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5989307-0754-4796-a7f4-79a20accfbd5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5989307-0754-4796-a7f4-79a20accfbd5-istiod-ca-cert\") pod \"a5989307-0754-4796-a7f4-79a20accfbd5\" (UID: \"a5989307-0754-4796-a7f4-79a20accfbd5\") "
Oct 30 22:03:23.883261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.883199    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc3e1a16-31d6-4f92-b201-0672d2b02946-istiod-ca-cert" pod "fc3e1a16-31d6-4f92-b201-0672d2b02946" (UID: "fc3e1a16-31d6-4f92-b201-0672d2b02946"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:23.883423 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.883310    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-podinfo\") pod \"fc3e1a16-31d6-4f92-b201-0672d2b02946\" (UID: \"fc3e1a16-31d6-4f92-b201-0672d2b02946\") "
Oct 30 22:03:23.883423 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.883336    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-workload-socket" pod "fc3e1a16-31d6-4f92-b201-0672d2b02946" (UID: "fc3e1a16-31d6-4f92-b201-0672d2b02946"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.883529 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.883454    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5989307-0754-4796-a7f4-79a20accfbd5-istiod-ca-cert" pod "a5989307-0754-4796-a7f4-79a20accfbd5" (UID: "a5989307-0754-4796-a7f4-79a20accfbd5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:23.883584 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.883560    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-workload-certs" pod "fc3e1a16-31d6-4f92-b201-0672d2b02946" (UID: "fc3e1a16-31d6-4f92-b201-0672d2b02946"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.883584 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.883576    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-workload-certs" pod "a5989307-0754-4796-a7f4-79a20accfbd5" (UID: "a5989307-0754-4796-a7f4-79a20accfbd5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.883792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.883771    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-data" pod "fc3e1a16-31d6-4f92-b201-0672d2b02946" (UID: "fc3e1a16-31d6-4f92-b201-0672d2b02946"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.883792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.883783    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-istio-data" pod "a5989307-0754-4796-a7f4-79a20accfbd5" (UID: "a5989307-0754-4796-a7f4-79a20accfbd5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.884486 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.884463    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-credential-socket" pod "a5989307-0754-4796-a7f4-79a20accfbd5" (UID: "a5989307-0754-4796-a7f4-79a20accfbd5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.884611 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.884527    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5989307-0754-4796-a7f4-79a20accfbd5-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.884611 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.884549    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc3e1a16-31d6-4f92-b201-0672d2b02946-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.884611 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.884564    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.884611 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.884579    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.884611 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.884594    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.884611 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.884608    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.884935 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.884623    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.884935 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.884632    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-workload-socket" pod "a5989307-0754-4796-a7f4-79a20accfbd5" (UID: "a5989307-0754-4796-a7f4-79a20accfbd5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.884935 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.884638    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.888812 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.888782    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-podinfo" pod "fc3e1a16-31d6-4f92-b201-0672d2b02946" (UID: "fc3e1a16-31d6-4f92-b201-0672d2b02946"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:03:23.888931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.888865    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5989307-0754-4796-a7f4-79a20accfbd5-kube-api-access-7bq68" pod "a5989307-0754-4796-a7f4-79a20accfbd5" (UID: "a5989307-0754-4796-a7f4-79a20accfbd5"). InnerVolumeSpecName "kube-api-access-7bq68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:23.889001 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.888964    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc3e1a16-31d6-4f92-b201-0672d2b02946-kube-api-access-7bw7n" pod "fc3e1a16-31d6-4f92-b201-0672d2b02946" (UID: "fc3e1a16-31d6-4f92-b201-0672d2b02946"). InnerVolumeSpecName "kube-api-access-7bw7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:23.889149 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.889123    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5989307-0754-4796-a7f4-79a20accfbd5-istio-token" pod "a5989307-0754-4796-a7f4-79a20accfbd5" (UID: "a5989307-0754-4796-a7f4-79a20accfbd5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:23.889380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.889357    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5989307-0754-4796-a7f4-79a20accfbd5-istio-podinfo" pod "a5989307-0754-4796-a7f4-79a20accfbd5" (UID: "a5989307-0754-4796-a7f4-79a20accfbd5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:03:23.889660 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.889637    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-token" pod "fc3e1a16-31d6-4f92-b201-0672d2b02946" (UID: "fc3e1a16-31d6-4f92-b201-0672d2b02946"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:23.889735 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.889717    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-envoy" pod "fc3e1a16-31d6-4f92-b201-0672d2b02946" (UID: "fc3e1a16-31d6-4f92-b201-0672d2b02946"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.890366 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.890347    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-istio-envoy" pod "a5989307-0754-4796-a7f4-79a20accfbd5" (UID: "a5989307-0754-4796-a7f4-79a20accfbd5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:23.986131 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.985816    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bq68\" (UniqueName: \"kubernetes.io/projected/a5989307-0754-4796-a7f4-79a20accfbd5-kube-api-access-7bq68\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.986131 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.985855    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.986131 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.985872    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.986131 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.985931    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5989307-0754-4796-a7f4-79a20accfbd5-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.986131 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.985954    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.986131 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.985977    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bw7n\" (UniqueName: \"kubernetes.io/projected/fc3e1a16-31d6-4f92-b201-0672d2b02946-kube-api-access-7bw7n\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.986131 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.986000    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc3e1a16-31d6-4f92-b201-0672d2b02946-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.986131 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.986021    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.986131 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.986041    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5989307-0754-4796-a7f4-79a20accfbd5-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:23.986131 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:23.986060    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5989307-0754-4796-a7f4-79a20accfbd5-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:24.101286 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.101197    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"
Oct 30 22:03:24.102803 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.102035    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"
Oct 30 22:03:24.153556 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.153530    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"]
Oct 30 22:03:24.213943 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.213907    5586 generic.go:356] "Generic (PLEG): container finished" podID="a5989307-0754-4796-a7f4-79a20accfbd5" containerID="ba6830afd8567063ccff765c08fd4bbc3cc2e495eb8f2f8af91d5bc3051a5f41" exitCode=0
Oct 30 22:03:24.214151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.213971    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"
Oct 30 22:03:24.214151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.213981    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5" event={"ID":"a5989307-0754-4796-a7f4-79a20accfbd5","Type":"ContainerDied","Data":"ba6830afd8567063ccff765c08fd4bbc3cc2e495eb8f2f8af91d5bc3051a5f41"}
Oct 30 22:03:24.214151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.214038    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5" event={"ID":"a5989307-0754-4796-a7f4-79a20accfbd5","Type":"ContainerDied","Data":"9f652a611a3902b5ee6d673f34c23f9c49e108a2b3cb1d19db313ce25b35d52c"}
Oct 30 22:03:24.214151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.214063    5586 scope.go:122] "RemoveContainer" containerID="ba6830afd8567063ccff765c08fd4bbc3cc2e495eb8f2f8af91d5bc3051a5f41"
Oct 30 22:03:24.216294 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.216264    5586 generic.go:356] "Generic (PLEG): container finished" podID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerID="fccc81d858c2daaf1cd4641d0d91067b25781a41ff3a6a42686a94f678efd869" exitCode=0
Oct 30 22:03:24.216405 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.216377    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" event={"ID":"fc3e1a16-31d6-4f92-b201-0672d2b02946","Type":"ContainerDied","Data":"fccc81d858c2daaf1cd4641d0d91067b25781a41ff3a6a42686a94f678efd869"}
Oct 30 22:03:24.216451 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.216420    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq" event={"ID":"fc3e1a16-31d6-4f92-b201-0672d2b02946","Type":"ContainerDied","Data":"773e41055fd16f1f707b5d7786b42986d8b16d5a47433f20d78ad78eede64a26"}
Oct 30 22:03:24.216505 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.216492    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"
Oct 30 22:03:24.225511 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.225395    5586 generic.go:356] "Generic (PLEG): container finished" podID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerID="029c8bac358e1267ce2cc358cf70a088509f3eeb3d58e079761a769b5648d89c" exitCode=0
Oct 30 22:03:24.225511 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.225466    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"
Oct 30 22:03:24.225511 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.225502    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" event={"ID":"a2297dba-5385-44d4-b0ae-1e470b525fc9","Type":"ContainerDied","Data":"029c8bac358e1267ce2cc358cf70a088509f3eeb3d58e079761a769b5648d89c"}
Oct 30 22:03:24.225783 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.225554    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw" event={"ID":"a2297dba-5385-44d4-b0ae-1e470b525fc9","Type":"ContainerDied","Data":"1a06ac9c03e79075e5c031ec4f8e2455efec17885f7063dcf3e64b603f968974"}
Oct 30 22:03:24.226696 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.226660    5586 scope.go:122] "RemoveContainer" containerID="ba6830afd8567063ccff765c08fd4bbc3cc2e495eb8f2f8af91d5bc3051a5f41"
Oct 30 22:03:24.227276 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:24.226920    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba6830afd8567063ccff765c08fd4bbc3cc2e495eb8f2f8af91d5bc3051a5f41\": container with ID starting with ba6830afd8567063ccff765c08fd4bbc3cc2e495eb8f2f8af91d5bc3051a5f41 not found: ID does not exist" containerID="ba6830afd8567063ccff765c08fd4bbc3cc2e495eb8f2f8af91d5bc3051a5f41"
Oct 30 22:03:24.227276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.226954    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba6830afd8567063ccff765c08fd4bbc3cc2e495eb8f2f8af91d5bc3051a5f41"} err="failed to get container status \"ba6830afd8567063ccff765c08fd4bbc3cc2e495eb8f2f8af91d5bc3051a5f41\": rpc error: code = NotFound desc = could not find container \"ba6830afd8567063ccff765c08fd4bbc3cc2e495eb8f2f8af91d5bc3051a5f41\": container with ID starting with ba6830afd8567063ccff765c08fd4bbc3cc2e495eb8f2f8af91d5bc3051a5f41 not found: ID does not exist"
Oct 30 22:03:24.227276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.226974    5586 scope.go:122] "RemoveContainer" containerID="fccc81d858c2daaf1cd4641d0d91067b25781a41ff3a6a42686a94f678efd869"
Oct 30 22:03:24.228024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.227998    5586 generic.go:356] "Generic (PLEG): container finished" podID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerID="2dd946dde2ce3cd9b36290df05ea6bf79b6994af1421bbf5b7c9afaf6cba5ff5" exitCode=0
Oct 30 22:03:24.228225 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.228206    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" event={"ID":"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd","Type":"ContainerDied","Data":"2dd946dde2ce3cd9b36290df05ea6bf79b6994af1421bbf5b7c9afaf6cba5ff5"}
Oct 30 22:03:24.228392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.228296    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"
Oct 30 22:03:24.228498 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.228473    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4" event={"ID":"77c5d7c9-2c19-4df8-a2ff-2e159881e1fd","Type":"ContainerDied","Data":"0d749c7f87afeef85c70ca1475a31595ccffbb2119dd26a1a89d564580c62454"}
Oct 30 22:03:24.280084 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.280059    5586 scope.go:122] "RemoveContainer" containerID="fccc81d858c2daaf1cd4641d0d91067b25781a41ff3a6a42686a94f678efd869"
Oct 30 22:03:24.291290 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:24.291256    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fccc81d858c2daaf1cd4641d0d91067b25781a41ff3a6a42686a94f678efd869\": container with ID starting with fccc81d858c2daaf1cd4641d0d91067b25781a41ff3a6a42686a94f678efd869 not found: ID does not exist" containerID="fccc81d858c2daaf1cd4641d0d91067b25781a41ff3a6a42686a94f678efd869"
Oct 30 22:03:24.291520 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.291492    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fccc81d858c2daaf1cd4641d0d91067b25781a41ff3a6a42686a94f678efd869"} err="failed to get container status \"fccc81d858c2daaf1cd4641d0d91067b25781a41ff3a6a42686a94f678efd869\": rpc error: code = NotFound desc = could not find container \"fccc81d858c2daaf1cd4641d0d91067b25781a41ff3a6a42686a94f678efd869\": container with ID starting with fccc81d858c2daaf1cd4641d0d91067b25781a41ff3a6a42686a94f678efd869 not found: ID does not exist"
Oct 30 22:03:24.291646 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.291635    5586 scope.go:122] "RemoveContainer" containerID="029c8bac358e1267ce2cc358cf70a088509f3eeb3d58e079761a769b5648d89c"
Oct 30 22:03:24.291866 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.291412    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp" event={"ID":"5a469237-ca99-4126-938b-bcbee6d9835f","Type":"ContainerStarted","Data":"61296f0b1f1e5b0e82e375d3543f944919c958ebd999c4674be00279dd681c69"}
Oct 30 22:03:24.311560 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.311483    5586 scope.go:122] "RemoveContainer" containerID="029c8bac358e1267ce2cc358cf70a088509f3eeb3d58e079761a769b5648d89c"
Oct 30 22:03:24.312337 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:24.312282    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"029c8bac358e1267ce2cc358cf70a088509f3eeb3d58e079761a769b5648d89c\": container with ID starting with 029c8bac358e1267ce2cc358cf70a088509f3eeb3d58e079761a769b5648d89c not found: ID does not exist" containerID="029c8bac358e1267ce2cc358cf70a088509f3eeb3d58e079761a769b5648d89c"
Oct 30 22:03:24.312797 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.312477    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"029c8bac358e1267ce2cc358cf70a088509f3eeb3d58e079761a769b5648d89c"} err="failed to get container status \"029c8bac358e1267ce2cc358cf70a088509f3eeb3d58e079761a769b5648d89c\": rpc error: code = NotFound desc = could not find container \"029c8bac358e1267ce2cc358cf70a088509f3eeb3d58e079761a769b5648d89c\": container with ID starting with 029c8bac358e1267ce2cc358cf70a088509f3eeb3d58e079761a769b5648d89c not found: ID does not exist"
Oct 30 22:03:24.312797 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.312516    5586 scope.go:122] "RemoveContainer" containerID="2dd946dde2ce3cd9b36290df05ea6bf79b6994af1421bbf5b7c9afaf6cba5ff5"
Oct 30 22:03:24.324334 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.324199    5586 scope.go:122] "RemoveContainer" containerID="2dd946dde2ce3cd9b36290df05ea6bf79b6994af1421bbf5b7c9afaf6cba5ff5"
Oct 30 22:03:24.324630 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:24.324576    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dd946dde2ce3cd9b36290df05ea6bf79b6994af1421bbf5b7c9afaf6cba5ff5\": container with ID starting with 2dd946dde2ce3cd9b36290df05ea6bf79b6994af1421bbf5b7c9afaf6cba5ff5 not found: ID does not exist" containerID="2dd946dde2ce3cd9b36290df05ea6bf79b6994af1421bbf5b7c9afaf6cba5ff5"
Oct 30 22:03:24.324741 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.324642    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dd946dde2ce3cd9b36290df05ea6bf79b6994af1421bbf5b7c9afaf6cba5ff5"} err="failed to get container status \"2dd946dde2ce3cd9b36290df05ea6bf79b6994af1421bbf5b7c9afaf6cba5ff5\": rpc error: code = NotFound desc = could not find container \"2dd946dde2ce3cd9b36290df05ea6bf79b6994af1421bbf5b7c9afaf6cba5ff5\": container with ID starting with 2dd946dde2ce3cd9b36290df05ea6bf79b6994af1421bbf5b7c9afaf6cba5ff5 not found: ID does not exist"
Oct 30 22:03:24.478258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.478196    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"]
Oct 30 22:03:24.484909 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.484860    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-qgjj4"]
Oct 30 22:03:24.590411 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.590315    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"]
Oct 30 22:03:24.597637 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.597599    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-hzntw"]
Oct 30 22:03:24.715272 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.715218    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"]
Oct 30 22:03:24.715874 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.715850    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-9cmqq"]
Oct 30 22:03:24.816527 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.815973    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"]
Oct 30 22:03:24.821912 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:24.821870    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-zxnt5"]
Oct 30 22:03:25.304667 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:25.304572    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp" event={"ID":"5a469237-ca99-4126-938b-bcbee6d9835f","Type":"ContainerStarted","Data":"874a063057efb52e3dcf20403b8524f04c1500bdae90c8bad641f14dd8e23c09"}
Oct 30 22:03:25.304843 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:25.304719    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:25.451820 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:25.451745    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp" podStartSLOduration=2.593648539 podStartE2EDuration="3.451722883s" totalImagesPullingTime="858.074344ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:25.4514981 +0000 UTC m=+1748.123402036" watchObservedRunningTime="2026-10-30 22:03:25.451722883 +0000 UTC m=+1748.123626819"
Oct 30 22:03:25.456682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:25.456648    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" path="/var/lib/kubelet/pods/77c5d7c9-2c19-4df8-a2ff-2e159881e1fd/volumes"
Oct 30 22:03:25.457299 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:25.457275    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" path="/var/lib/kubelet/pods/a2297dba-5385-44d4-b0ae-1e470b525fc9/volumes"
Oct 30 22:03:25.457821 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:25.457801    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5989307-0754-4796-a7f4-79a20accfbd5" path="/var/lib/kubelet/pods/a5989307-0754-4796-a7f4-79a20accfbd5/volumes"
Oct 30 22:03:25.458715 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:25.458693    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" path="/var/lib/kubelet/pods/fc3e1a16-31d6-4f92-b201-0672d2b02946/volumes"
Oct 30 22:03:25.503068 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:25.503025    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:27.796207 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:27.796160    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:28.447997 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:28.447958    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:28.656276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:28.656216    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:29.584735 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:29.584692    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:31.319013 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:31.318985    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:03:33.291094 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:33.291055    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:34.403397 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:34.403291    5586 generic.go:356] "Generic (PLEG): container finished" podID="08f613b9-8231-4e10-8a70-5dee789fc9f7" containerID="c1f958f7ae0aa57c8a5553f56d2d8e13cffbca941725d8bfedd50fc9a083fdb1" exitCode=0
Oct 30 22:03:34.403397 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:34.403366    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp" event={"ID":"08f613b9-8231-4e10-8a70-5dee789fc9f7","Type":"ContainerDied","Data":"c1f958f7ae0aa57c8a5553f56d2d8e13cffbca941725d8bfedd50fc9a083fdb1"}
Oct 30 22:03:34.403992 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:34.403973    5586 scope.go:122] "RemoveContainer" containerID="c1f958f7ae0aa57c8a5553f56d2d8e13cffbca941725d8bfedd50fc9a083fdb1"
Oct 30 22:03:34.405733 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:34.405704    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:34.405853 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:34.405776    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:34.405853 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:34.405808    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:03:35.413163 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:35.413123    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp" event={"ID":"08f613b9-8231-4e10-8a70-5dee789fc9f7","Type":"ContainerStarted","Data":"15c3e146d1d909f20a7aa68356811347214756397e7946f7cc4a656850c1f6b6"}
Oct 30 22:03:35.446453 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:35.446412    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:03:35.972463 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:35.972418    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:03:35.974063 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:35.974031    5586 patch_prober.go:29] interesting pod/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 30 22:03:35.974185 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:35.974101    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp" podUID="08f613b9-8231-4e10-8a70-5dee789fc9f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 30 22:03:36.973123 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:36.973087    5586 patch_prober.go:29] interesting pod/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 30 22:03:36.973568 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:36.973153    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp" podUID="08f613b9-8231-4e10-8a70-5dee789fc9f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 30 22:03:37.972602 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:37.972561    5586 patch_prober.go:29] interesting pod/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 30 22:03:37.972827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:37.972627    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp" podUID="08f613b9-8231-4e10-8a70-5dee789fc9f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 30 22:03:38.544257 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:38.543278    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:38.972907 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:38.972874    5586 patch_prober.go:29] interesting pod/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 30 22:03:38.973090 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:38.972930    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp" podUID="08f613b9-8231-4e10-8a70-5dee789fc9f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 30 22:03:39.972576 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:39.972542    5586 patch_prober.go:29] interesting pod/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused" start-of-body=
Oct 30 22:03:39.973058 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:39.972605    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp" podUID="08f613b9-8231-4e10-8a70-5dee789fc9f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.108:15021/healthz/ready\": dial tcp 10.134.0.108:15021: connect: connection refused"
Oct 30 22:03:40.484912 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:40.484868    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:40.977798 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:40.977765    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:03:40.978964 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:40.978933    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:03:41.427366 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:41.427329    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:41.684813 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:41.684707    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:42.116173 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:42.116077    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:43.714332 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.714293    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"]
Oct 30 22:03:43.714976 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.714962    5586 cpu_manager.go:414] "containerMap: removing container" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy"
Oct 30 22:03:43.715031 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.714977    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy"
Oct 30 22:03:43.715031 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.714995    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a5989307-0754-4796-a7f4-79a20accfbd5" containerName="istio-proxy"
Oct 30 22:03:43.715031 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.715001    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5989307-0754-4796-a7f4-79a20accfbd5" containerName="istio-proxy"
Oct 30 22:03:43.715031 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.715027    5586 cpu_manager.go:414] "containerMap: removing container" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy"
Oct 30 22:03:43.715168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.715033    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy"
Oct 30 22:03:43.715168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.715045    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy"
Oct 30 22:03:43.715168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.715050    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy"
Oct 30 22:03:43.715168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.715159    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc3e1a16-31d6-4f92-b201-0672d2b02946" containerName="istio-proxy"
Oct 30 22:03:43.715309 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.715171    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5989307-0754-4796-a7f4-79a20accfbd5" containerName="istio-proxy"
Oct 30 22:03:43.715309 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.715184    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="77c5d7c9-2c19-4df8-a2ff-2e159881e1fd" containerName="istio-proxy"
Oct 30 22:03:43.715309 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.715192    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2297dba-5385-44d4-b0ae-1e470b525fc9" containerName="istio-proxy"
Oct 30 22:03:43.717519 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.717501    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:43.764066 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.764026    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-maas-api-tls\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:43.764260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.764097    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdhrl\" (UniqueName: \"kubernetes.io/projected/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-kube-api-access-xdhrl\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:43.764260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.764178    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-openshift-service-ca\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:43.764342 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.764307    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-metrics-certs\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:43.770404 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.770387    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6df3ba0e-serving-cert\""
Oct 30 22:03:43.779166 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.779138    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"]
Oct 30 22:03:43.865603 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.865559    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-maas-api-tls\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:43.865808 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.865656    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdhrl\" (UniqueName: \"kubernetes.io/projected/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-kube-api-access-xdhrl\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:43.865808 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.865682    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-openshift-service-ca\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:43.865808 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:43.865743    5586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-serving-cert: secret "maas-api-e2e-mt-6df3ba0e-serving-cert" not found
Oct 30 22:03:43.866001 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:43.865824    5586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-maas-api-tls podName:603aeab9-aac6-4ed9-8336-08f7c2acdd6c nodeName:}" failed. No retries permitted until 2026-10-30 22:03:44.365798588 +0000 UTC m=+1767.037702517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-maas-api-tls") pod "maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4" (UID: "603aeab9-aac6-4ed9-8336-08f7c2acdd6c") : secret "maas-api-e2e-mt-6df3ba0e-serving-cert" not found
Oct 30 22:03:43.866127 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.865750    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-metrics-certs\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:43.866659 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.866637    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-openshift-service-ca\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:43.868594 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.868569    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-metrics-certs\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:43.929125 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.929087    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdhrl\" (UniqueName: \"kubernetes.io/projected/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-kube-api-access-xdhrl\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:43.930882 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:43.930845    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:44.182309 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:44.182266    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:44.373655 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:44.373621    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-maas-api-tls\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:44.376363 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:44.376332    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-maas-api-tls\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:44.630389 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:44.630298    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:45.021084 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:45.021053    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"]
Oct 30 22:03:45.022057 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:03:45.022024    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod603aeab9_aac6_4ed9_8336_08f7c2acdd6c.slice/crio-6d7dda6db40bea92f67591b954e3421a4dc3524abf0fc8dc870913d9db792279 WatchSource:0}: Error finding container 6d7dda6db40bea92f67591b954e3421a4dc3524abf0fc8dc870913d9db792279: Status 404 returned error can't find the container with id 6d7dda6db40bea92f67591b954e3421a4dc3524abf0fc8dc870913d9db792279
Oct 30 22:03:45.515154 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:45.515116    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4" event={"ID":"603aeab9-aac6-4ed9-8336-08f7c2acdd6c","Type":"ContainerStarted","Data":"6d7dda6db40bea92f67591b954e3421a4dc3524abf0fc8dc870913d9db792279"}
Oct 30 22:03:46.523842 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:46.523799    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4" event={"ID":"603aeab9-aac6-4ed9-8336-08f7c2acdd6c","Type":"ContainerStarted","Data":"a5570b864e8a4ebaea90e8d28c39ecf1cef4a51132f8e5373a6434b869c5389f"}
Oct 30 22:03:46.524374 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:46.523913    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:46.650885 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:46.650818    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4" podStartSLOduration=2.705629958 podStartE2EDuration="3.650797069s" totalImagesPullingTime="945.167111ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:46.648772874 +0000 UTC m=+1769.320676821" watchObservedRunningTime="2026-10-30 22:03:46.650797069 +0000 UTC m=+1769.322701006"
Oct 30 22:03:48.362398 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:03:48.362354    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:03:49.291921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:49.291881    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:49.844098 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:49.844057    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:52.541461 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:52.541428    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:03:54.139309 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:54.139267    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:59.009816 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:59.009767    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:59.294220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:59.294115    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:03:59.743824 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:03:59.743784    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:00.260152 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.260110    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"]
Oct 30 22:04:00.263661 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.263635    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.306255 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.305384    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-data\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.306255 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.305502    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv8k4\" (UniqueName: \"kubernetes.io/projected/c0960ade-ab00-437c-84a2-2c9e702a9d06-kube-api-access-pv8k4\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.306255 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.305549    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-workload-socket\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.306255 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.305617    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-credential-socket\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.306255 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.305671    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-token\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.306255 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.305739    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0960ade-ab00-437c-84a2-2c9e702a9d06-istiod-ca-cert\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.306255 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.305768    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-workload-certs\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.306255 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.305811    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-envoy\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.306255 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.305849    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-podinfo\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.365425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.365388    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-20c7c7-openshift-default-dockercfg-drdn8\""
Oct 30 22:04:00.376931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.376883    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"]
Oct 30 22:04:00.407600 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.407559    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-workload-socket\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.407811 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.407675    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-credential-socket\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.407811 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.407743    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-token\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.407949 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.407827    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0960ade-ab00-437c-84a2-2c9e702a9d06-istiod-ca-cert\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.407949 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.407863    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-workload-certs\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.407949 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.407910    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-envoy\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.408119 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.407950    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-podinfo\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.408119 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.407979    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-workload-socket\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.408119 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.408022    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-data\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.408119 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.408111    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv8k4\" (UniqueName: \"kubernetes.io/projected/c0960ade-ab00-437c-84a2-2c9e702a9d06-kube-api-access-pv8k4\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.408839 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.408810    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-workload-certs\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.408949 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.408843    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-credential-socket\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.409024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.408994    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-data\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.409292 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.408712    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0960ade-ab00-437c-84a2-2c9e702a9d06-istiod-ca-cert\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.412180 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.411959    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-podinfo\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.412180 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.412103    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-envoy\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.442691 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.442590    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:00.484020 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.483982    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-token\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.488046 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.488012    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv8k4\" (UniqueName: \"kubernetes.io/projected/c0960ade-ab00-437c-84a2-2c9e702a9d06-kube-api-access-pv8k4\") pod \"e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") " pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:00.581036 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:00.580925    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:01.040010 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:01.039981    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"]
Oct 30 22:04:01.040921 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:04:01.040877    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0960ade_ab00_437c_84a2_2c9e702a9d06.slice/crio-591aaf2fc458c8f2bbe22ce36a1e657d302221fdf3fa3d23a93ab606e488af6b WatchSource:0}: Error finding container 591aaf2fc458c8f2bbe22ce36a1e657d302221fdf3fa3d23a93ab606e488af6b: Status 404 returned error can't find the container with id 591aaf2fc458c8f2bbe22ce36a1e657d302221fdf3fa3d23a93ab606e488af6b
Oct 30 22:04:01.043886 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:01.043849    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:01.043952 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:01.043927    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:01.043991 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:01.043959    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:01.056506 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:01.056461    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:01.657841 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:01.657801    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7" event={"ID":"c0960ade-ab00-437c-84a2-2c9e702a9d06","Type":"ContainerStarted","Data":"ff9499ce9242c4dc1ef67356bbfaef27d506b1be8433c43b4cdf3fef4a87332c"}
Oct 30 22:04:01.658308 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:01.657864    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7" event={"ID":"c0960ade-ab00-437c-84a2-2c9e702a9d06","Type":"ContainerStarted","Data":"591aaf2fc458c8f2bbe22ce36a1e657d302221fdf3fa3d23a93ab606e488af6b"}
Oct 30 22:04:01.779023 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:01.778954    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7" podStartSLOduration=1.778933681 podStartE2EDuration="1.778933681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:01.776620326 +0000 UTC m=+1784.448524261" watchObservedRunningTime="2026-10-30 22:04:01.778933681 +0000 UTC m=+1784.450837617"
Oct 30 22:04:02.581193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:02.581158    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:02.587529 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:02.587496    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:02.665184 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:02.665148    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:02.666193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:02.666170    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:04:03.572524 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:03.561111    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"]
Oct 30 22:04:03.572524 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:03.571638    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr" podUID="1a027617-89f1-4e5e-84f0-4bc510c9b357" containerName="istio-proxy" containerID="cri-o://b0626bfc4b5fb425254a26a649f25184b685cab13989321c89b1cea86c3adfab" gracePeriod=30
Oct 30 22:04:04.359168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:04.359116    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:04.681024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:04.680982    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"]
Oct 30 22:04:04.681304 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:04.681275    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" podUID="8310bcd1-2853-49f2-9618-0ff100c794bd" containerName="istio-proxy" containerID="cri-o://001f9ab0b437f7fc4d717fa2ab5b30583ea73c6aca9a3f5fbaa002d8ac7effab" gracePeriod=30
Oct 30 22:04:04.694359 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:04.694325    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"]
Oct 30 22:04:04.695351 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:04.695275    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" podUID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" containerName="istio-proxy" containerID="cri-o://cfbfed66ebe801b59031c5fbdb440fd36351dea81229e09ec5b1d675d544f147" gracePeriod=30
Oct 30 22:04:04.702471 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:04.702436    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"]
Oct 30 22:04:04.702702 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:04.702664    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" podUID="146b8994-5857-43e6-a00e-40683229192f" containerName="istio-proxy" containerID="cri-o://f2830b4a27ce15d036f400720112f600d17d967c89e685068da4c44e2e1f657a" gracePeriod=30
Oct 30 22:04:04.784108 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:04.784028    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"]
Oct 30 22:04:04.784519 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:04.784457    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy" containerID="cri-o://a33ea16f9a593ba8a1b8397b2571f32e61fff0cb063262db54ac27938a9dcf05" gracePeriod=30
Oct 30 22:04:04.785427 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:04.785029    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:04.968319 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:04.967942    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:07.076338 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:07.076301    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:04:07.076906 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:07.076881    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" podUID="146b8994-5857-43e6-a00e-40683229192f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:04:08.017297 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.017184    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:08.254643 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.254600    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:08.258276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.257924    5586 patch_prober.go:29] interesting pod/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:04:08.258276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.257986    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" podUID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:04:08.444393 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.444332    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"]
Oct 30 22:04:08.463496 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.463451    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"
Oct 30 22:04:08.490021 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.489989    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"]
Oct 30 22:04:08.490502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.490475    5586 patch_prober.go:29] interesting pod/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:04:08.490572 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.490535    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr" podUID="1a027617-89f1-4e5e-84f0-4bc510c9b357" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:04:08.536270 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.533664    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-20c7c7-dockercfg-hhfs8\""
Oct 30 22:04:08.536270 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.533738    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-20c7c7\""
Oct 30 22:04:08.538436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.536636    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0245bc-1651-4789-9944-5430e821cd76-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg\" (UID: \"0b0245bc-1651-4789-9944-5430e821cd76\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"
Oct 30 22:04:08.538436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.536696    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tntr\" (UniqueName: \"kubernetes.io/projected/0b0245bc-1651-4789-9944-5430e821cd76-kube-api-access-7tntr\") pod \"payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg\" (UID: \"0b0245bc-1651-4789-9944-5430e821cd76\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"
Oct 30 22:04:08.583276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.582767    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"]
Oct 30 22:04:08.588952 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.587053    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"
Oct 30 22:04:08.636826 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.636756    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"]
Oct 30 22:04:08.637745 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.637715    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tntr\" (UniqueName: \"kubernetes.io/projected/0b0245bc-1651-4789-9944-5430e821cd76-kube-api-access-7tntr\") pod \"payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg\" (UID: \"0b0245bc-1651-4789-9944-5430e821cd76\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"
Oct 30 22:04:08.639041 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.639005    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0245bc-1651-4789-9944-5430e821cd76-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg\" (UID: \"0b0245bc-1651-4789-9944-5430e821cd76\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"
Oct 30 22:04:08.639945 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.639918    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0245bc-1651-4789-9944-5430e821cd76-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg\" (UID: \"0b0245bc-1651-4789-9944-5430e821cd76\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"
Oct 30 22:04:08.712797 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.712762    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tntr\" (UniqueName: \"kubernetes.io/projected/0b0245bc-1651-4789-9944-5430e821cd76-kube-api-access-7tntr\") pod \"payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg\" (UID: \"0b0245bc-1651-4789-9944-5430e821cd76\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"
Oct 30 22:04:08.768405 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.768361    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmnrb\" (UniqueName: \"kubernetes.io/projected/0909fb48-36b5-4cd0-b3e6-f026073e3557-kube-api-access-jmnrb\") pod \"payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94\" (UID: \"0909fb48-36b5-4cd0-b3e6-f026073e3557\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"
Oct 30 22:04:08.768612 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.768583    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0909fb48-36b5-4cd0-b3e6-f026073e3557-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94\" (UID: \"0909fb48-36b5-4cd0-b3e6-f026073e3557\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"
Oct 30 22:04:08.779390 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.779048    5586 generic.go:356] "Generic (PLEG): container finished" podID="1a027617-89f1-4e5e-84f0-4bc510c9b357" containerID="b0626bfc4b5fb425254a26a649f25184b685cab13989321c89b1cea86c3adfab" exitCode=0
Oct 30 22:04:08.779390 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.779311    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr" event={"ID":"1a027617-89f1-4e5e-84f0-4bc510c9b357","Type":"ContainerDied","Data":"b0626bfc4b5fb425254a26a649f25184b685cab13989321c89b1cea86c3adfab"}
Oct 30 22:04:08.787838 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.787530    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"
Oct 30 22:04:08.872478 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.871289    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmnrb\" (UniqueName: \"kubernetes.io/projected/0909fb48-36b5-4cd0-b3e6-f026073e3557-kube-api-access-jmnrb\") pod \"payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94\" (UID: \"0909fb48-36b5-4cd0-b3e6-f026073e3557\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"
Oct 30 22:04:08.872478 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.871422    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0909fb48-36b5-4cd0-b3e6-f026073e3557-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94\" (UID: \"0909fb48-36b5-4cd0-b3e6-f026073e3557\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"
Oct 30 22:04:08.872478 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.872138    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0909fb48-36b5-4cd0-b3e6-f026073e3557-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94\" (UID: \"0909fb48-36b5-4cd0-b3e6-f026073e3557\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"
Oct 30 22:04:08.931708 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.931390    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmnrb\" (UniqueName: \"kubernetes.io/projected/0909fb48-36b5-4cd0-b3e6-f026073e3557-kube-api-access-jmnrb\") pod \"payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94\" (UID: \"0909fb48-36b5-4cd0-b3e6-f026073e3557\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"
Oct 30 22:04:08.941786 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:08.941503    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 22:04:09.022276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.022158    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"
Oct 30 22:04:09.074228 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.074190    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-envoy\") pod \"1a027617-89f1-4e5e-84f0-4bc510c9b357\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") "
Oct 30 22:04:09.074469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.074365    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a027617-89f1-4e5e-84f0-4bc510c9b357-kube-api-access-2cjc2\" (UniqueName: \"kubernetes.io/projected/1a027617-89f1-4e5e-84f0-4bc510c9b357-kube-api-access-2cjc2\") pod \"1a027617-89f1-4e5e-84f0-4bc510c9b357\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") "
Oct 30 22:04:09.074469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.074411    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-workload-socket\") pod \"1a027617-89f1-4e5e-84f0-4bc510c9b357\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") "
Oct 30 22:04:09.074614 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.074492    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-token\" (UniqueName: \"kubernetes.io/projected/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-token\") pod \"1a027617-89f1-4e5e-84f0-4bc510c9b357\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") "
Oct 30 22:04:09.074614 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.074549    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-data\") pod \"1a027617-89f1-4e5e-84f0-4bc510c9b357\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") "
Oct 30 22:04:09.074614 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.074586    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a027617-89f1-4e5e-84f0-4bc510c9b357-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a027617-89f1-4e5e-84f0-4bc510c9b357-istiod-ca-cert\") pod \"1a027617-89f1-4e5e-84f0-4bc510c9b357\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") "
Oct 30 22:04:09.074727 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.074638    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-credential-socket\") pod \"1a027617-89f1-4e5e-84f0-4bc510c9b357\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") "
Oct 30 22:04:09.074727 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.074719    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-workload-certs\") pod \"1a027617-89f1-4e5e-84f0-4bc510c9b357\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") "
Oct 30 22:04:09.074791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.074777    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-podinfo\") pod \"1a027617-89f1-4e5e-84f0-4bc510c9b357\" (UID: \"1a027617-89f1-4e5e-84f0-4bc510c9b357\") "
Oct 30 22:04:09.075409 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.075105    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-data" pod "1a027617-89f1-4e5e-84f0-4bc510c9b357" (UID: "1a027617-89f1-4e5e-84f0-4bc510c9b357"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:09.075409 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.075177    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-credential-socket" pod "1a027617-89f1-4e5e-84f0-4bc510c9b357" (UID: "1a027617-89f1-4e5e-84f0-4bc510c9b357"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:09.075576 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.075436    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-workload-certs" pod "1a027617-89f1-4e5e-84f0-4bc510c9b357" (UID: "1a027617-89f1-4e5e-84f0-4bc510c9b357"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:09.075576 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.075475    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:09.075576 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.075498    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:09.075867 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.075773    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-workload-socket" pod "1a027617-89f1-4e5e-84f0-4bc510c9b357" (UID: "1a027617-89f1-4e5e-84f0-4bc510c9b357"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:09.075867 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.075776    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a027617-89f1-4e5e-84f0-4bc510c9b357-istiod-ca-cert" pod "1a027617-89f1-4e5e-84f0-4bc510c9b357" (UID: "1a027617-89f1-4e5e-84f0-4bc510c9b357"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:04:09.078023 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.077987    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-podinfo" pod "1a027617-89f1-4e5e-84f0-4bc510c9b357" (UID: "1a027617-89f1-4e5e-84f0-4bc510c9b357"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:04:09.078531 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.078500    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-envoy" pod "1a027617-89f1-4e5e-84f0-4bc510c9b357" (UID: "1a027617-89f1-4e5e-84f0-4bc510c9b357"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:09.079697 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.079670    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-token" pod "1a027617-89f1-4e5e-84f0-4bc510c9b357" (UID: "1a027617-89f1-4e5e-84f0-4bc510c9b357"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:09.082478 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.082451    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a027617-89f1-4e5e-84f0-4bc510c9b357-kube-api-access-2cjc2" pod "1a027617-89f1-4e5e-84f0-4bc510c9b357" (UID: "1a027617-89f1-4e5e-84f0-4bc510c9b357"). InnerVolumeSpecName "kube-api-access-2cjc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:09.180639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.180594    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cjc2\" (UniqueName: \"kubernetes.io/projected/1a027617-89f1-4e5e-84f0-4bc510c9b357-kube-api-access-2cjc2\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:09.180639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.180640    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:09.180965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.180653    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:09.180965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.180665    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a027617-89f1-4e5e-84f0-4bc510c9b357-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:09.180965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.180677    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:09.180965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.180688    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:09.180965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.180700    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a027617-89f1-4e5e-84f0-4bc510c9b357-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:09.218465 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.218429    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"]
Oct 30 22:04:09.219984 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:04:09.219946    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b0245bc_1651_4789_9944_5430e821cd76.slice/crio-27b877bb1b6ac1c159c54347fd8ec46e9deb5e178d2d0788e8f67c2fabd83783 WatchSource:0}: Error finding container 27b877bb1b6ac1c159c54347fd8ec46e9deb5e178d2d0788e8f67c2fabd83783: Status 404 returned error can't find the container with id 27b877bb1b6ac1c159c54347fd8ec46e9deb5e178d2d0788e8f67c2fabd83783
Oct 30 22:04:09.377423 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.377375    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:04:09.377807 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.377443    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:09.444318 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:04:09.444279    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0909fb48_36b5_4cd0_b3e6_f026073e3557.slice/crio-037ed3971e34fb40f63ef879d6ea2b5f2cefa566c7d4ba210005daa74205151a WatchSource:0}: Error finding container 037ed3971e34fb40f63ef879d6ea2b5f2cefa566c7d4ba210005daa74205151a: Status 404 returned error can't find the container with id 037ed3971e34fb40f63ef879d6ea2b5f2cefa566c7d4ba210005daa74205151a
Oct 30 22:04:09.449698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.449671    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"]
Oct 30 22:04:09.547740 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.547653    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"]
Oct 30 22:04:09.548764 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.548739    5586 cpu_manager.go:414] "containerMap: removing container" podUID="1a027617-89f1-4e5e-84f0-4bc510c9b357" containerName="istio-proxy"
Oct 30 22:04:09.548764 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.548764    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a027617-89f1-4e5e-84f0-4bc510c9b357" containerName="istio-proxy"
Oct 30 22:04:09.548942 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.548929    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a027617-89f1-4e5e-84f0-4bc510c9b357" containerName="istio-proxy"
Oct 30 22:04:09.551874 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.551850    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:09.608156 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.607739    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-ea3cb2-serving-cert\""
Oct 30 22:04:09.614354 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.614315    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"]
Oct 30 22:04:09.688845 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.688809    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:09.689017 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.688975    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fca9942-5704-4346-8c4f-abe2c9bfefe2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:09.689093 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.689068    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt9nb\" (UniqueName: \"kubernetes.io/projected/5fca9942-5704-4346-8c4f-abe2c9bfefe2-kube-api-access-wt9nb\") pod \"maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:09.689171 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.689112    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:09.762112 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.762064    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:09.803395 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.792678    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:09.803395 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.792825    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fca9942-5704-4346-8c4f-abe2c9bfefe2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:09.803395 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.792911    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt9nb\" (UniqueName: \"kubernetes.io/projected/5fca9942-5704-4346-8c4f-abe2c9bfefe2-kube-api-access-wt9nb\") pod \"maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:09.803395 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.792965    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:09.803395 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.794279    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fca9942-5704-4346-8c4f-abe2c9bfefe2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:09.803395 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:04:09.794394    5586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-serving-cert: secret "maas-api-e2e-shared-a-w2-ea3cb2-serving-cert" not found
Oct 30 22:04:09.803395 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:04:09.794455    5586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-maas-api-tls podName:5fca9942-5704-4346-8c4f-abe2c9bfefe2 nodeName:}" failed. No retries permitted until 2026-10-30 22:04:10.294435448 +0000 UTC m=+1792.966339369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-maas-api-tls") pod "maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q" (UID: "5fca9942-5704-4346-8c4f-abe2c9bfefe2") : secret "maas-api-e2e-shared-a-w2-ea3cb2-serving-cert" not found
Oct 30 22:04:09.809569 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.809497    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:09.819297 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.812716    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr" event={"ID":"1a027617-89f1-4e5e-84f0-4bc510c9b357","Type":"ContainerDied","Data":"35c44cc302927b5483389545e6475d33ec820bf16a40ff9a7e98b9bccd08cd38"}
Oct 30 22:04:09.819297 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.812793    5586 scope.go:122] "RemoveContainer" containerID="b0626bfc4b5fb425254a26a649f25184b685cab13989321c89b1cea86c3adfab"
Oct 30 22:04:09.819297 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.812788    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"
Oct 30 22:04:09.821114 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.819683    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg" event={"ID":"0b0245bc-1651-4789-9944-5430e821cd76","Type":"ContainerStarted","Data":"90091a8e801156ff86322abe3cb638d62840cac2713f4ee9e1912e14633ab200"}
Oct 30 22:04:09.821114 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.819766    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg" event={"ID":"0b0245bc-1651-4789-9944-5430e821cd76","Type":"ContainerStarted","Data":"27b877bb1b6ac1c159c54347fd8ec46e9deb5e178d2d0788e8f67c2fabd83783"}
Oct 30 22:04:09.821114 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.819799    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"
Oct 30 22:04:09.833223 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.832857    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94" event={"ID":"0909fb48-36b5-4cd0-b3e6-f026073e3557","Type":"ContainerStarted","Data":"2a8347ad62aee63ce027eede03a31115307414fabe483cd4f399a0c71e80df9f"}
Oct 30 22:04:09.833223 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.832912    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94" event={"ID":"0909fb48-36b5-4cd0-b3e6-f026073e3557","Type":"ContainerStarted","Data":"037ed3971e34fb40f63ef879d6ea2b5f2cefa566c7d4ba210005daa74205151a"}
Oct 30 22:04:09.834203 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.833735    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"
Oct 30 22:04:09.854923 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.854871    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt9nb\" (UniqueName: \"kubernetes.io/projected/5fca9942-5704-4346-8c4f-abe2c9bfefe2-kube-api-access-wt9nb\") pod \"maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:09.988305 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:09.988206    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94" podStartSLOduration=1.9881846570000001 podStartE2EDuration="1.988184657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:09.983746451 +0000 UTC m=+1792.655650391" watchObservedRunningTime="2026-10-30 22:04:09.988184657 +0000 UTC m=+1792.660088595"
Oct 30 22:04:10.038161 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.038079    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:10.124301 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.124114    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg" podStartSLOduration=2.124093728 podStartE2EDuration="2.124093728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:10.107436127 +0000 UTC m=+1792.779340073" watchObservedRunningTime="2026-10-30 22:04:10.124093728 +0000 UTC m=+1792.795997664"
Oct 30 22:04:10.169437 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.169338    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 22:04:10.235030 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.234985    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"]
Oct 30 22:04:10.241893 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.241861    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-rjwxr"]
Oct 30 22:04:10.254659 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.254533    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:10.258924 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.258897    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 22:04:10.266418 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.266394    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 22:04:10.278914 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.278884    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 22:04:10.314288 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.314252    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/146b8994-5857-43e6-a00e-40683229192f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/146b8994-5857-43e6-a00e-40683229192f-istio-podinfo\") pod \"146b8994-5857-43e6-a00e-40683229192f\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") "
Oct 30 22:04:10.314464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.314395    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-workload-certs\") pod \"146b8994-5857-43e6-a00e-40683229192f\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") "
Oct 30 22:04:10.314464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.314438    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/146b8994-5857-43e6-a00e-40683229192f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/146b8994-5857-43e6-a00e-40683229192f-istiod-ca-cert\") pod \"146b8994-5857-43e6-a00e-40683229192f\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") "
Oct 30 22:04:10.314604 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.314489    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-workload-socket\") pod \"146b8994-5857-43e6-a00e-40683229192f\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") "
Oct 30 22:04:10.314604 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.314534    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-istio-data\") pod \"146b8994-5857-43e6-a00e-40683229192f\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") "
Oct 30 22:04:10.314713 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.314637    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/146b8994-5857-43e6-a00e-40683229192f-kube-api-access-nk46v\" (UniqueName: \"kubernetes.io/projected/146b8994-5857-43e6-a00e-40683229192f-kube-api-access-nk46v\") pod \"146b8994-5857-43e6-a00e-40683229192f\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") "
Oct 30 22:04:10.314770 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.314718    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-workload-certs" pod "146b8994-5857-43e6-a00e-40683229192f" (UID: "146b8994-5857-43e6-a00e-40683229192f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.314826 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.314773    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/146b8994-5857-43e6-a00e-40683229192f-istio-token\" (UniqueName: \"kubernetes.io/projected/146b8994-5857-43e6-a00e-40683229192f-istio-token\") pod \"146b8994-5857-43e6-a00e-40683229192f\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") "
Oct 30 22:04:10.314826 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.314805    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-credential-socket\") pod \"146b8994-5857-43e6-a00e-40683229192f\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") "
Oct 30 22:04:10.314924 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.314893    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-istio-envoy\") pod \"146b8994-5857-43e6-a00e-40683229192f\" (UID: \"146b8994-5857-43e6-a00e-40683229192f\") "
Oct 30 22:04:10.314924 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.314896    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-istio-data" pod "146b8994-5857-43e6-a00e-40683229192f" (UID: "146b8994-5857-43e6-a00e-40683229192f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.315039 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.314947    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/146b8994-5857-43e6-a00e-40683229192f-istiod-ca-cert" pod "146b8994-5857-43e6-a00e-40683229192f" (UID: "146b8994-5857-43e6-a00e-40683229192f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:04:10.315147 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.315123    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-workload-socket" pod "146b8994-5857-43e6-a00e-40683229192f" (UID: "146b8994-5857-43e6-a00e-40683229192f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.315205 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.315164    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:10.315205 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.315183    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-credential-socket" pod "146b8994-5857-43e6-a00e-40683229192f" (UID: "146b8994-5857-43e6-a00e-40683229192f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.315736 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.315631    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.315736 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.315658    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/146b8994-5857-43e6-a00e-40683229192f-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.315736 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.315673    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.315736 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.315688    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.315736 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.315704    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.317496 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.317466    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/146b8994-5857-43e6-a00e-40683229192f-kube-api-access-nk46v" pod "146b8994-5857-43e6-a00e-40683229192f" (UID: "146b8994-5857-43e6-a00e-40683229192f"). InnerVolumeSpecName "kube-api-access-nk46v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:10.317713 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.317691    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/146b8994-5857-43e6-a00e-40683229192f-istio-podinfo" pod "146b8994-5857-43e6-a00e-40683229192f" (UID: "146b8994-5857-43e6-a00e-40683229192f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:04:10.318915 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.318892    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:10.319202 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.319180    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/146b8994-5857-43e6-a00e-40683229192f-istio-token" pod "146b8994-5857-43e6-a00e-40683229192f" (UID: "146b8994-5857-43e6-a00e-40683229192f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:10.320732 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.320711    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-istio-envoy" pod "146b8994-5857-43e6-a00e-40683229192f" (UID: "146b8994-5857-43e6-a00e-40683229192f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.416522 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.416417    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-workload-certs\") pod \"b40a8985-4520-4dbc-894d-b07e94d9deac\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.416558    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-podinfo\") pod \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.416594    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-envoy\") pod \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.417896    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b40a8985-4520-4dbc-894d-b07e94d9deac-kube-api-access-26xxh\" (UniqueName: \"kubernetes.io/projected/b40a8985-4520-4dbc-894d-b07e94d9deac-kube-api-access-26xxh\") pod \"b40a8985-4520-4dbc-894d-b07e94d9deac\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.417809    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-workload-certs" pod "b40a8985-4520-4dbc-894d-b07e94d9deac" (UID: "b40a8985-4520-4dbc-894d-b07e94d9deac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.417936    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istiod-ca-cert\") pod \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.417978    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8310bcd1-2853-49f2-9618-0ff100c794bd-kube-api-access-6wh2d\" (UniqueName: \"kubernetes.io/projected/8310bcd1-2853-49f2-9618-0ff100c794bd-kube-api-access-6wh2d\") pod \"8310bcd1-2853-49f2-9618-0ff100c794bd\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418027    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-workload-socket\") pod \"8310bcd1-2853-49f2-9618-0ff100c794bd\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418077    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-credential-socket\") pod \"8310bcd1-2853-49f2-9618-0ff100c794bd\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418105    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b40a8985-4520-4dbc-894d-b07e94d9deac-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b40a8985-4520-4dbc-894d-b07e94d9deac-istiod-ca-cert\") pod \"b40a8985-4520-4dbc-894d-b07e94d9deac\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418169    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-envoy\") pod \"8310bcd1-2853-49f2-9618-0ff100c794bd\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418219    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-workload-socket\") pod \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418304    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-kube-api-access-l7j6d\" (UniqueName: \"kubernetes.io/projected/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-kube-api-access-l7j6d\") pod \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418355    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-credential-socket\") pod \"b40a8985-4520-4dbc-894d-b07e94d9deac\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418385    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-podinfo\") pod \"8310bcd1-2853-49f2-9618-0ff100c794bd\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418439    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8310bcd1-2853-49f2-9618-0ff100c794bd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8310bcd1-2853-49f2-9618-0ff100c794bd-istiod-ca-cert\") pod \"8310bcd1-2853-49f2-9618-0ff100c794bd\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418477    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-data\") pod \"8310bcd1-2853-49f2-9618-0ff100c794bd\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418519    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-envoy\") pod \"b40a8985-4520-4dbc-894d-b07e94d9deac\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418570    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-podinfo\") pod \"b40a8985-4520-4dbc-894d-b07e94d9deac\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418605    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-workload-socket\") pod \"b40a8985-4520-4dbc-894d-b07e94d9deac\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418631    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-data\") pod \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418704    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-token\" (UniqueName: \"kubernetes.io/projected/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-token\") pod \"8310bcd1-2853-49f2-9618-0ff100c794bd\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418787    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-workload-certs\") pod \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418819    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-workload-certs\") pod \"8310bcd1-2853-49f2-9618-0ff100c794bd\" (UID: \"8310bcd1-2853-49f2-9618-0ff100c794bd\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418870    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-credential-socket\") pod \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") "
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418884    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-workload-socket" pod "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" (UID: "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.418903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418899    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-token\" (UniqueName: \"kubernetes.io/projected/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-token\") pod \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\" (UID: \"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3\") "
Oct 30 22:04:10.420467 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.418977    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-token\" (UniqueName: \"kubernetes.io/projected/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-token\") pod \"b40a8985-4520-4dbc-894d-b07e94d9deac\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") "
Oct 30 22:04:10.420467 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.419068    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-data\") pod \"b40a8985-4520-4dbc-894d-b07e94d9deac\" (UID: \"b40a8985-4520-4dbc-894d-b07e94d9deac\") "
Oct 30 22:04:10.420467 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.419704    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-credential-socket" pod "8310bcd1-2853-49f2-9618-0ff100c794bd" (UID: "8310bcd1-2853-49f2-9618-0ff100c794bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.420467 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.419937    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-workload-socket" pod "8310bcd1-2853-49f2-9618-0ff100c794bd" (UID: "8310bcd1-2853-49f2-9618-0ff100c794bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.420467 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.420423    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-data" pod "b40a8985-4520-4dbc-894d-b07e94d9deac" (UID: "b40a8985-4520-4dbc-894d-b07e94d9deac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.423517 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.421045    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk46v\" (UniqueName: \"kubernetes.io/projected/146b8994-5857-43e6-a00e-40683229192f-kube-api-access-nk46v\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.423517 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.422275    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-workload-socket" pod "b40a8985-4520-4dbc-894d-b07e94d9deac" (UID: "b40a8985-4520-4dbc-894d-b07e94d9deac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.423517 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.422492    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-credential-socket" pod "b40a8985-4520-4dbc-894d-b07e94d9deac" (UID: "b40a8985-4520-4dbc-894d-b07e94d9deac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.423517 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.422524    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b40a8985-4520-4dbc-894d-b07e94d9deac-istiod-ca-cert" pod "b40a8985-4520-4dbc-894d-b07e94d9deac" (UID: "b40a8985-4520-4dbc-894d-b07e94d9deac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:04:10.423517 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.422809    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-workload-certs" pod "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" (UID: "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.423517 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.423082    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-data" pod "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" (UID: "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.423517 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.423259    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-workload-certs" pod "8310bcd1-2853-49f2-9618-0ff100c794bd" (UID: "8310bcd1-2853-49f2-9618-0ff100c794bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.423517 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.423493    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-credential-socket" pod "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" (UID: "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.424031 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.423683    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-data" pod "8310bcd1-2853-49f2-9618-0ff100c794bd" (UID: "8310bcd1-2853-49f2-9618-0ff100c794bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.424031 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.423961    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8310bcd1-2853-49f2-9618-0ff100c794bd-istiod-ca-cert" pod "8310bcd1-2853-49f2-9618-0ff100c794bd" (UID: "8310bcd1-2853-49f2-9618-0ff100c794bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:04:10.424772 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.421072    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.424887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.424788    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.424887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.424805    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/146b8994-5857-43e6-a00e-40683229192f-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.424887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.424816    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.424887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.424829    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.424887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.424843    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/146b8994-5857-43e6-a00e-40683229192f-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.424887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.424856    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.424887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.424868    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/146b8994-5857-43e6-a00e-40683229192f-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.425319 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.425065    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istiod-ca-cert" pod "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" (UID: "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:04:10.425551 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.425500    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-token" pod "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" (UID: "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:10.426502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.426457    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-kube-api-access-l7j6d" pod "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" (UID: "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3"). InnerVolumeSpecName "kube-api-access-l7j6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:10.426981 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.426947    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-podinfo" pod "8310bcd1-2853-49f2-9618-0ff100c794bd" (UID: "8310bcd1-2853-49f2-9618-0ff100c794bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:04:10.427081 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.427007    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b40a8985-4520-4dbc-894d-b07e94d9deac-kube-api-access-26xxh" pod "b40a8985-4520-4dbc-894d-b07e94d9deac" (UID: "b40a8985-4520-4dbc-894d-b07e94d9deac"). InnerVolumeSpecName "kube-api-access-26xxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:10.427140 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.427117    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8310bcd1-2853-49f2-9618-0ff100c794bd-kube-api-access-6wh2d" pod "8310bcd1-2853-49f2-9618-0ff100c794bd" (UID: "8310bcd1-2853-49f2-9618-0ff100c794bd"). InnerVolumeSpecName "kube-api-access-6wh2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:10.427337 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.427312    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-podinfo" pod "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" (UID: "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:04:10.427419 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.427380    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-token" pod "b40a8985-4520-4dbc-894d-b07e94d9deac" (UID: "b40a8985-4520-4dbc-894d-b07e94d9deac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:10.427542 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.427513    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-envoy" pod "8310bcd1-2853-49f2-9618-0ff100c794bd" (UID: "8310bcd1-2853-49f2-9618-0ff100c794bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.427673 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.427600    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-token" pod "8310bcd1-2853-49f2-9618-0ff100c794bd" (UID: "8310bcd1-2853-49f2-9618-0ff100c794bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:10.427908 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.427882    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-envoy" pod "b40a8985-4520-4dbc-894d-b07e94d9deac" (UID: "b40a8985-4520-4dbc-894d-b07e94d9deac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.428586 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.428562    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-podinfo" pod "b40a8985-4520-4dbc-894d-b07e94d9deac" (UID: "b40a8985-4520-4dbc-894d-b07e94d9deac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:04:10.429070 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.429052    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-envoy" pod "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" (UID: "5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:10.471827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.471782    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.526922    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7j6d\" (UniqueName: \"kubernetes.io/projected/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-kube-api-access-l7j6d\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.526958    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.526975    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.526989    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8310bcd1-2853-49f2-9618-0ff100c794bd-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527004    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527017    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527033    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527047    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b40a8985-4520-4dbc-894d-b07e94d9deac-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527062    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527075    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527088    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527101    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527115    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527130    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527145    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b40a8985-4520-4dbc-894d-b07e94d9deac-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527158    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527174    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527188    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26xxh\" (UniqueName: \"kubernetes.io/projected/b40a8985-4520-4dbc-894d-b07e94d9deac-kube-api-access-26xxh\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527205    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527218    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wh2d\" (UniqueName: \"kubernetes.io/projected/8310bcd1-2853-49f2-9618-0ff100c794bd-kube-api-access-6wh2d\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527249    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b40a8985-4520-4dbc-894d-b07e94d9deac-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.534310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.527262    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8310bcd1-2853-49f2-9618-0ff100c794bd-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:10.853087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.853038    5586 generic.go:356] "Generic (PLEG): container finished" podID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" containerID="cfbfed66ebe801b59031c5fbdb440fd36351dea81229e09ec5b1d675d544f147" exitCode=0
Oct 30 22:04:10.853087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.853093    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"
Oct 30 22:04:10.853402 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.853114    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" event={"ID":"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3","Type":"ContainerDied","Data":"cfbfed66ebe801b59031c5fbdb440fd36351dea81229e09ec5b1d675d544f147"}
Oct 30 22:04:10.853402 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.853173    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh" event={"ID":"5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3","Type":"ContainerDied","Data":"e816a22aa8f118bc00d5bb6338c93cd4b57b7320a842e58c606197dee38bb9b9"}
Oct 30 22:04:10.853402 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.853205    5586 scope.go:122] "RemoveContainer" containerID="cfbfed66ebe801b59031c5fbdb440fd36351dea81229e09ec5b1d675d544f147"
Oct 30 22:04:10.856207 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.856179    5586 generic.go:356] "Generic (PLEG): container finished" podID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerID="a33ea16f9a593ba8a1b8397b2571f32e61fff0cb063262db54ac27938a9dcf05" exitCode=0
Oct 30 22:04:10.856933 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.856360    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" event={"ID":"b40a8985-4520-4dbc-894d-b07e94d9deac","Type":"ContainerDied","Data":"a33ea16f9a593ba8a1b8397b2571f32e61fff0cb063262db54ac27938a9dcf05"}
Oct 30 22:04:10.856933 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.856397    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"
Oct 30 22:04:10.856933 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.856419    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6" event={"ID":"b40a8985-4520-4dbc-894d-b07e94d9deac","Type":"ContainerDied","Data":"28dd7b9caa8ba0a4015e92be34232b58cc1bc65be68e8c88c93036d0720b8dd0"}
Oct 30 22:04:10.860400 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.858279    5586 generic.go:356] "Generic (PLEG): container finished" podID="8310bcd1-2853-49f2-9618-0ff100c794bd" containerID="001f9ab0b437f7fc4d717fa2ab5b30583ea73c6aca9a3f5fbaa002d8ac7effab" exitCode=0
Oct 30 22:04:10.860400 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.858494    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" event={"ID":"8310bcd1-2853-49f2-9618-0ff100c794bd","Type":"ContainerDied","Data":"001f9ab0b437f7fc4d717fa2ab5b30583ea73c6aca9a3f5fbaa002d8ac7effab"}
Oct 30 22:04:10.860400 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.858549    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s" event={"ID":"8310bcd1-2853-49f2-9618-0ff100c794bd","Type":"ContainerDied","Data":"ed3dc2fa25e15612c083835749f23e2c906f9ba1bd36614d802240350967b16c"}
Oct 30 22:04:10.860400 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.858628    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"
Oct 30 22:04:10.865087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.864852    5586 generic.go:356] "Generic (PLEG): container finished" podID="146b8994-5857-43e6-a00e-40683229192f" containerID="f2830b4a27ce15d036f400720112f600d17d967c89e685068da4c44e2e1f657a" exitCode=0
Oct 30 22:04:10.865087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.864896    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" event={"ID":"146b8994-5857-43e6-a00e-40683229192f","Type":"ContainerDied","Data":"f2830b4a27ce15d036f400720112f600d17d967c89e685068da4c44e2e1f657a"}
Oct 30 22:04:10.865087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.864949    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"
Oct 30 22:04:10.865087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.864964    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz" event={"ID":"146b8994-5857-43e6-a00e-40683229192f","Type":"ContainerDied","Data":"d9664c54caeadc9351e1203e3657846989e98e5a34f4fb698edd4eb779163db9"}
Oct 30 22:04:10.870740 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.870714    5586 scope.go:122] "RemoveContainer" containerID="cfbfed66ebe801b59031c5fbdb440fd36351dea81229e09ec5b1d675d544f147"
Oct 30 22:04:10.871103 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:04:10.871073    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfbfed66ebe801b59031c5fbdb440fd36351dea81229e09ec5b1d675d544f147\": container with ID starting with cfbfed66ebe801b59031c5fbdb440fd36351dea81229e09ec5b1d675d544f147 not found: ID does not exist" containerID="cfbfed66ebe801b59031c5fbdb440fd36351dea81229e09ec5b1d675d544f147"
Oct 30 22:04:10.871215 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.871109    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfbfed66ebe801b59031c5fbdb440fd36351dea81229e09ec5b1d675d544f147"} err="failed to get container status \"cfbfed66ebe801b59031c5fbdb440fd36351dea81229e09ec5b1d675d544f147\": rpc error: code = NotFound desc = could not find container \"cfbfed66ebe801b59031c5fbdb440fd36351dea81229e09ec5b1d675d544f147\": container with ID starting with cfbfed66ebe801b59031c5fbdb440fd36351dea81229e09ec5b1d675d544f147 not found: ID does not exist"
Oct 30 22:04:10.871215 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.871134    5586 scope.go:122] "RemoveContainer" containerID="a33ea16f9a593ba8a1b8397b2571f32e61fff0cb063262db54ac27938a9dcf05"
Oct 30 22:04:10.888355 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.888329    5586 scope.go:122] "RemoveContainer" containerID="67efb1ff70be31453fb0b03d24f694c7cf42d57cd50d0effc4a938b0eaf29e08"
Oct 30 22:04:10.899428 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.899401    5586 scope.go:122] "RemoveContainer" containerID="a33ea16f9a593ba8a1b8397b2571f32e61fff0cb063262db54ac27938a9dcf05"
Oct 30 22:04:10.899883 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:04:10.899861    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a33ea16f9a593ba8a1b8397b2571f32e61fff0cb063262db54ac27938a9dcf05\": container with ID starting with a33ea16f9a593ba8a1b8397b2571f32e61fff0cb063262db54ac27938a9dcf05 not found: ID does not exist" containerID="a33ea16f9a593ba8a1b8397b2571f32e61fff0cb063262db54ac27938a9dcf05"
Oct 30 22:04:10.900006 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.899895    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a33ea16f9a593ba8a1b8397b2571f32e61fff0cb063262db54ac27938a9dcf05"} err="failed to get container status \"a33ea16f9a593ba8a1b8397b2571f32e61fff0cb063262db54ac27938a9dcf05\": rpc error: code = NotFound desc = could not find container \"a33ea16f9a593ba8a1b8397b2571f32e61fff0cb063262db54ac27938a9dcf05\": container with ID starting with a33ea16f9a593ba8a1b8397b2571f32e61fff0cb063262db54ac27938a9dcf05 not found: ID does not exist"
Oct 30 22:04:10.900006 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.899923    5586 scope.go:122] "RemoveContainer" containerID="67efb1ff70be31453fb0b03d24f694c7cf42d57cd50d0effc4a938b0eaf29e08"
Oct 30 22:04:10.900287 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:04:10.900266    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67efb1ff70be31453fb0b03d24f694c7cf42d57cd50d0effc4a938b0eaf29e08\": container with ID starting with 67efb1ff70be31453fb0b03d24f694c7cf42d57cd50d0effc4a938b0eaf29e08 not found: ID does not exist" containerID="67efb1ff70be31453fb0b03d24f694c7cf42d57cd50d0effc4a938b0eaf29e08"
Oct 30 22:04:10.900382 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.900298    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67efb1ff70be31453fb0b03d24f694c7cf42d57cd50d0effc4a938b0eaf29e08"} err="failed to get container status \"67efb1ff70be31453fb0b03d24f694c7cf42d57cd50d0effc4a938b0eaf29e08\": rpc error: code = NotFound desc = could not find container \"67efb1ff70be31453fb0b03d24f694c7cf42d57cd50d0effc4a938b0eaf29e08\": container with ID starting with 67efb1ff70be31453fb0b03d24f694c7cf42d57cd50d0effc4a938b0eaf29e08 not found: ID does not exist"
Oct 30 22:04:10.900382 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.900318    5586 scope.go:122] "RemoveContainer" containerID="001f9ab0b437f7fc4d717fa2ab5b30583ea73c6aca9a3f5fbaa002d8ac7effab"
Oct 30 22:04:10.911298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.911266    5586 scope.go:122] "RemoveContainer" containerID="001f9ab0b437f7fc4d717fa2ab5b30583ea73c6aca9a3f5fbaa002d8ac7effab"
Oct 30 22:04:10.911705 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:04:10.911678    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"001f9ab0b437f7fc4d717fa2ab5b30583ea73c6aca9a3f5fbaa002d8ac7effab\": container with ID starting with 001f9ab0b437f7fc4d717fa2ab5b30583ea73c6aca9a3f5fbaa002d8ac7effab not found: ID does not exist" containerID="001f9ab0b437f7fc4d717fa2ab5b30583ea73c6aca9a3f5fbaa002d8ac7effab"
Oct 30 22:04:10.911799 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.911715    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"001f9ab0b437f7fc4d717fa2ab5b30583ea73c6aca9a3f5fbaa002d8ac7effab"} err="failed to get container status \"001f9ab0b437f7fc4d717fa2ab5b30583ea73c6aca9a3f5fbaa002d8ac7effab\": rpc error: code = NotFound desc = could not find container \"001f9ab0b437f7fc4d717fa2ab5b30583ea73c6aca9a3f5fbaa002d8ac7effab\": container with ID starting with 001f9ab0b437f7fc4d717fa2ab5b30583ea73c6aca9a3f5fbaa002d8ac7effab not found: ID does not exist"
Oct 30 22:04:10.911799 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.911734    5586 scope.go:122] "RemoveContainer" containerID="f2830b4a27ce15d036f400720112f600d17d967c89e685068da4c44e2e1f657a"
Oct 30 22:04:10.935186 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.935156    5586 scope.go:122] "RemoveContainer" containerID="f2830b4a27ce15d036f400720112f600d17d967c89e685068da4c44e2e1f657a"
Oct 30 22:04:10.935703 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:04:10.935674    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2830b4a27ce15d036f400720112f600d17d967c89e685068da4c44e2e1f657a\": container with ID starting with f2830b4a27ce15d036f400720112f600d17d967c89e685068da4c44e2e1f657a not found: ID does not exist" containerID="f2830b4a27ce15d036f400720112f600d17d967c89e685068da4c44e2e1f657a"
Oct 30 22:04:10.935791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.935717    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2830b4a27ce15d036f400720112f600d17d967c89e685068da4c44e2e1f657a"} err="failed to get container status \"f2830b4a27ce15d036f400720112f600d17d967c89e685068da4c44e2e1f657a\": rpc error: code = NotFound desc = could not find container \"f2830b4a27ce15d036f400720112f600d17d967c89e685068da4c44e2e1f657a\": container with ID starting with f2830b4a27ce15d036f400720112f600d17d967c89e685068da4c44e2e1f657a not found: ID does not exist"
Oct 30 22:04:10.947209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:10.947048    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"]
Oct 30 22:04:10.950754 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:04:10.950710    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fca9942_5704_4346_8c4f_abe2c9bfefe2.slice/crio-31553e6df30c943c5b8355e98d3d41460f3414b4c9a76a4c663cf17f219e60e3 WatchSource:0}: Error finding container 31553e6df30c943c5b8355e98d3d41460f3414b4c9a76a4c663cf17f219e60e3: Status 404 returned error can't find the container with id 31553e6df30c943c5b8355e98d3d41460f3414b4c9a76a4c663cf17f219e60e3
Oct 30 22:04:11.034685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:11.034640    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"]
Oct 30 22:04:11.053774 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:11.053736    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-5qpd6"]
Oct 30 22:04:11.162500 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:11.162447    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"]
Oct 30 22:04:11.166221 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:11.166180    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-ssx6s"]
Oct 30 22:04:11.277737 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:11.277649    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"]
Oct 30 22:04:11.281366 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:11.281322    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-wq9bh"]
Oct 30 22:04:11.389289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:11.389220    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"]
Oct 30 22:04:11.392638 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:11.392608    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc009f-openshift-default-645d5cbf67-2gplz"]
Oct 30 22:04:11.450276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:11.450223    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="146b8994-5857-43e6-a00e-40683229192f" path="/var/lib/kubelet/pods/146b8994-5857-43e6-a00e-40683229192f/volumes"
Oct 30 22:04:11.450947 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:11.450923    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a027617-89f1-4e5e-84f0-4bc510c9b357" path="/var/lib/kubelet/pods/1a027617-89f1-4e5e-84f0-4bc510c9b357/volumes"
Oct 30 22:04:11.451574 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:11.451557    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" path="/var/lib/kubelet/pods/5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3/volumes"
Oct 30 22:04:11.452402 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:11.452385    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8310bcd1-2853-49f2-9618-0ff100c794bd" path="/var/lib/kubelet/pods/8310bcd1-2853-49f2-9618-0ff100c794bd/volumes"
Oct 30 22:04:11.452840 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:11.452827    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" path="/var/lib/kubelet/pods/b40a8985-4520-4dbc-894d-b07e94d9deac/volumes"
Oct 30 22:04:11.878090 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:11.878049    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q" event={"ID":"5fca9942-5704-4346-8c4f-abe2c9bfefe2","Type":"ContainerStarted","Data":"31553e6df30c943c5b8355e98d3d41460f3414b4c9a76a4c663cf17f219e60e3"}
Oct 30 22:04:12.893351 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:12.893301    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q" event={"ID":"5fca9942-5704-4346-8c4f-abe2c9bfefe2","Type":"ContainerStarted","Data":"d32a0a1ecfa5f29b4095484466949a85f94865d655a3ac6eb58f4dd9c15ac22e"}
Oct 30 22:04:13.245687 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:12.893399    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:13.245687 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:13.012575    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q" podStartSLOduration=3.005540222 podStartE2EDuration="4.012558996s" totalImagesPullingTime="1.007018774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:13.009776229 +0000 UTC m=+1795.681680176" watchObservedRunningTime="2026-10-30 22:04:13.012558996 +0000 UTC m=+1795.684462931"
Oct 30 22:04:13.902620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:13.902526    5586 generic.go:356] "Generic (PLEG): container finished" podID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerID="d525a8b9d7cfa3cd8d201500ae8112c65218a8b7eb984ccb8324448ae8d0cdab" exitCode=0
Oct 30 22:04:13.902620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:13.902600    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" event={"ID":"8a89955b-47d7-4a3a-895f-6c3f69b695aa","Type":"ContainerDied","Data":"d525a8b9d7cfa3cd8d201500ae8112c65218a8b7eb984ccb8324448ae8d0cdab"}
Oct 30 22:04:13.903268 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:13.903227    5586 scope.go:122] "RemoveContainer" containerID="d525a8b9d7cfa3cd8d201500ae8112c65218a8b7eb984ccb8324448ae8d0cdab"
Oct 30 22:04:13.904800 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:13.904773    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:13.904881 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:13.904844    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:13.904881 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:13.904876    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:14.912628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:14.912585    5586 generic.go:356] "Generic (PLEG): container finished" podID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerID="e2373d3d6e42c85d745de5944cf1c7f7290636749099a83030a7a2c142bc01db" exitCode=0
Oct 30 22:04:14.913100 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:14.912653    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm" event={"ID":"a76b490b-c349-4a5c-ba7e-6445596e7ee0","Type":"ContainerDied","Data":"e2373d3d6e42c85d745de5944cf1c7f7290636749099a83030a7a2c142bc01db"}
Oct 30 22:04:14.913190 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:14.913172    5586 scope.go:122] "RemoveContainer" containerID="e2373d3d6e42c85d745de5944cf1c7f7290636749099a83030a7a2c142bc01db"
Oct 30 22:04:14.915076 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:14.915041    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:14.915179 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:14.915075    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" event={"ID":"8a89955b-47d7-4a3a-895f-6c3f69b695aa","Type":"ContainerStarted","Data":"881994a42458a3976ce76c6d80b1856d70db6638c4c2f4457c4fbf2ec417f995"}
Oct 30 22:04:14.915179 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:14.915135    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:14.915310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:14.915182    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:15.233324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:15.232681    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:04:15.233324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:15.232821    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:04:15.233324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:15.232865    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Oct 30 22:04:15.273768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:15.273723    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:15.933259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:15.932786    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm" event={"ID":"a76b490b-c349-4a5c-ba7e-6445596e7ee0","Type":"ContainerStarted","Data":"7447c2eb2d2afb6bc5c6e3a4cc34a9950fac8038474543d1a440a6098c7e9f48"}
Oct 30 22:04:16.065985 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:16.065942    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:04:16.066198 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:16.066080    5586 patch_prober.go:29] interesting pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:04:16.066198 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:16.066120    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm" podUID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 22:04:16.242320 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:16.238417    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:04:16.242320 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:16.238479    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Oct 30 22:04:17.070259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:17.069729    5586 patch_prober.go:29] interesting pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:04:17.070259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:17.069799    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm" podUID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 22:04:17.236259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:17.235493    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:04:17.236259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:17.235579    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Oct 30 22:04:17.811038 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:17.811003    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:04:17.812147 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:17.812128    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:04:17.817371 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:17.817346    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:04:17.818729 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:17.818713    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:04:18.066333 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:18.065731    5586 patch_prober.go:29] interesting pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:04:18.066333 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:18.065802    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm" podUID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 22:04:18.240259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:18.239699    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:04:18.240259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:18.239763    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Oct 30 22:04:18.395650 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:04:18.395547    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:04:18.908514 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:18.908486    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:04:19.065387 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.065348    5586 patch_prober.go:29] interesting pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:04:19.065670 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.065639    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm" podUID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 22:04:19.250319 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.249709    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:04:19.250319 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.249776    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Oct 30 22:04:19.686403 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.686365    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"]
Oct 30 22:04:19.687108 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687089    5586 cpu_manager.go:414] "containerMap: removing container" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy"
Oct 30 22:04:19.687165 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687109    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy"
Oct 30 22:04:19.687165 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687119    5586 cpu_manager.go:414] "containerMap: removing container" podUID="146b8994-5857-43e6-a00e-40683229192f" containerName="istio-proxy"
Oct 30 22:04:19.687165 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687123    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="146b8994-5857-43e6-a00e-40683229192f" containerName="istio-proxy"
Oct 30 22:04:19.687165 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687146    5586 cpu_manager.go:414] "containerMap: removing container" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy"
Oct 30 22:04:19.687165 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687151    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy"
Oct 30 22:04:19.687343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687169    5586 cpu_manager.go:414] "containerMap: removing container" podUID="8310bcd1-2853-49f2-9618-0ff100c794bd" containerName="istio-proxy"
Oct 30 22:04:19.687343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687174    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8310bcd1-2853-49f2-9618-0ff100c794bd" containerName="istio-proxy"
Oct 30 22:04:19.687343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687184    5586 cpu_manager.go:414] "containerMap: removing container" podUID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" containerName="istio-proxy"
Oct 30 22:04:19.687343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687188    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" containerName="istio-proxy"
Oct 30 22:04:19.687343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687335    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy"
Oct 30 22:04:19.687492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687348    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="8310bcd1-2853-49f2-9618-0ff100c794bd" containerName="istio-proxy"
Oct 30 22:04:19.687492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687357    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="5eba1a2d-9bc2-4a7e-9840-e3071d7cf0f3" containerName="istio-proxy"
Oct 30 22:04:19.687492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687371    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="b40a8985-4520-4dbc-894d-b07e94d9deac" containerName="istio-proxy"
Oct 30 22:04:19.687492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.687377    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="146b8994-5857-43e6-a00e-40683229192f" containerName="istio-proxy"
Oct 30 22:04:19.690136 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.690117    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:19.762416 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.762380    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w6-9d4043-serving-cert\""
Oct 30 22:04:19.766259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.764863    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"]
Oct 30 22:04:19.818760 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.818717    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae3b993-2c09-41f3-bcc7-eec79cae6198-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:19.818966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.818946    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:19.819104 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.819088    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mlqz\" (UniqueName: \"kubernetes.io/projected/1ae3b993-2c09-41f3-bcc7-eec79cae6198-kube-api-access-8mlqz\") pod \"maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:19.819809 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.819764    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:19.944316 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.921090    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae3b993-2c09-41f3-bcc7-eec79cae6198-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:19.944316 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.921252    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:19.944316 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.921346    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mlqz\" (UniqueName: \"kubernetes.io/projected/1ae3b993-2c09-41f3-bcc7-eec79cae6198-kube-api-access-8mlqz\") pod \"maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:19.944316 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.921429    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:19.944316 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:04:19.921652    5586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-serving-cert: secret "maas-api-e2e-shared-a-w6-9d4043-serving-cert" not found
Oct 30 22:04:19.944316 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:04:19.921716    5586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-maas-api-tls podName:1ae3b993-2c09-41f3-bcc7-eec79cae6198 nodeName:}" failed. No retries permitted until 2026-10-30 22:04:20.421695245 +0000 UTC m=+1803.093599161 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-maas-api-tls") pod "maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc" (UID: "1ae3b993-2c09-41f3-bcc7-eec79cae6198") : secret "maas-api-e2e-shared-a-w6-9d4043-serving-cert" not found
Oct 30 22:04:19.944316 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.935646    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:19.945668 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.945633    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae3b993-2c09-41f3-bcc7-eec79cae6198-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:19.987262 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.979974    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:20.002832 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:19.999424    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mlqz\" (UniqueName: \"kubernetes.io/projected/1ae3b993-2c09-41f3-bcc7-eec79cae6198-kube-api-access-8mlqz\") pod \"maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:20.066270 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:20.058145    5586 patch_prober.go:29] interesting pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:04:20.067163 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:20.058218    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm" podUID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 22:04:20.099942 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:20.099906    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:20.238089 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:20.237959    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:04:20.238427 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:20.238374    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Oct 30 22:04:20.449841 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:20.449796    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:20.453899 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:20.453863    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:20.603632 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:20.603530    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:20.869420 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:20.869339    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"
Oct 30 22:04:20.869420 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:20.869410    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"
Oct 30 22:04:21.049262 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:04:21.019946    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ae3b993_2c09_41f3_bcc7_eec79cae6198.slice/crio-7c872e1032e4d442e00abb4e530460cbb00889470aa7252795ee76603c4242b2 WatchSource:0}: Error finding container 7c872e1032e4d442e00abb4e530460cbb00889470aa7252795ee76603c4242b2: Status 404 returned error can't find the container with id 7c872e1032e4d442e00abb4e530460cbb00889470aa7252795ee76603c4242b2
Oct 30 22:04:21.059763 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:21.059724    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"]
Oct 30 22:04:21.060462 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:21.060335    5586 patch_prober.go:29] interesting pod/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 22:04:21.060462 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:21.060388    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm" podUID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 22:04:21.067301 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:21.064444    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc" event={"ID":"1ae3b993-2c09-41f3-bcc7-eec79cae6198","Type":"ContainerStarted","Data":"7c872e1032e4d442e00abb4e530460cbb00889470aa7252795ee76603c4242b2"}
Oct 30 22:04:21.239756 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:21.239723    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:04:21.240494 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:21.240466    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:04:21.241610 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:21.241590    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:04:22.063218 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:22.063181    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:04:22.063707 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:22.063688    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:04:22.064576 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:22.064557    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:04:22.073839 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:22.073807    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc" event={"ID":"1ae3b993-2c09-41f3-bcc7-eec79cae6198","Type":"ContainerStarted","Data":"a16d46e69f8d20d7c2144a47ed2acc5582b4d8ca1908f0c2940408aeef0daf0e"}
Oct 30 22:04:22.073954 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:22.073883    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:22.445059 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:22.444986    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc" podStartSLOduration=2.576743304 podStartE2EDuration="3.444968902s" totalImagesPullingTime="868.225598ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:22.313992859 +0000 UTC m=+1804.985896794" watchObservedRunningTime="2026-10-30 22:04:22.444968902 +0000 UTC m=+1805.116872838"
Oct 30 22:04:23.473655 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:23.473620    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:24.348554 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.348518    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"]
Oct 30 22:04:24.351912 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.351892    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.445369 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.445330    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-9f8e92-openshift-default-dockercfg-ndxr5\""
Oct 30 22:04:24.459137 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.459107    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"]
Oct 30 22:04:24.518512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.517657    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-data\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.518512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.517722    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-podinfo\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.518512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.517867    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-workload-certs\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.518512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.517912    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istiod-ca-cert\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.518512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.518069    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-workload-socket\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.518512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.518168    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrln4\" (UniqueName: \"kubernetes.io/projected/02b0df12-ffa4-4f62-b12d-fd1e2808a730-kube-api-access-zrln4\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.518512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.518205    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-credential-socket\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.518512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.518277    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-token\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.518512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.518327    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-envoy\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.519784 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.518888    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:24.620184 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.620100    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-workload-socket\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.620378 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.620186    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrln4\" (UniqueName: \"kubernetes.io/projected/02b0df12-ffa4-4f62-b12d-fd1e2808a730-kube-api-access-zrln4\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.620378 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.620211    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-credential-socket\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.620378 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.620275    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-token\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.620378 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.620322    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-envoy\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.620579 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.620394    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-data\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.620579 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.620417    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-podinfo\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.620579 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.620488    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-workload-certs\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.620579 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.620522    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istiod-ca-cert\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.620579 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.620552    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-workload-socket\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.620819 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.620572    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-credential-socket\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.620930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.620909    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-data\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.620964 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.620926    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-workload-certs\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.621331 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.621314    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istiod-ca-cert\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.623347 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.623327    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-envoy\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.623493 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.623473    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-podinfo\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.677309 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.677277    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-token\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.677939 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.677919    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrln4\" (UniqueName: \"kubernetes.io/projected/02b0df12-ffa4-4f62-b12d-fd1e2808a730-kube-api-access-zrln4\") pod \"e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") " pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:24.902795 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.902706    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:24.965097 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:24.965059    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:25.303307 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:25.303262    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:25.413689 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:25.413653    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"]
Oct 30 22:04:25.414168 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:04:25.414139    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02b0df12_ffa4_4f62_b12d_fd1e2808a730.slice/crio-3661f2445f64da003c4dbbc87672486f7dd2d06c2e77984edcb3fca1b984f4e4 WatchSource:0}: Error finding container 3661f2445f64da003c4dbbc87672486f7dd2d06c2e77984edcb3fca1b984f4e4: Status 404 returned error can't find the container with id 3661f2445f64da003c4dbbc87672486f7dd2d06c2e77984edcb3fca1b984f4e4
Oct 30 22:04:25.417138 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:25.417100    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:25.417289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:25.417189    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:25.417289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:25.417257    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:25.990366 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:25.990332    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:26.110914 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:26.110876    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw" event={"ID":"02b0df12-ffa4-4f62-b12d-fd1e2808a730","Type":"ContainerStarted","Data":"7b783f94d3d7052180811ae2d6114920f2987a6e2c6aec2c0eba42d09fde7c2c"}
Oct 30 22:04:26.111139 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:26.110960    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw" event={"ID":"02b0df12-ffa4-4f62-b12d-fd1e2808a730","Type":"ContainerStarted","Data":"3661f2445f64da003c4dbbc87672486f7dd2d06c2e77984edcb3fca1b984f4e4"}
Oct 30 22:04:26.281625 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:26.281506    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw" podStartSLOduration=2.281491418 podStartE2EDuration="2.281491418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:26.278600743 +0000 UTC m=+1808.950504680" watchObservedRunningTime="2026-10-30 22:04:26.281491418 +0000 UTC m=+1808.953395352"
Oct 30 22:04:26.965825 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:26.965784    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:26.972291 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:26.972259    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:27.118284 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:27.118249    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:27.119250 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:27.119209    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:04:28.087827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:28.087794    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:04:29.160113 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:29.160018    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:29.349675 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:29.349621    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:29.563666 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:29.563552    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:30.042587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:30.042551    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:32.591214 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:32.591169    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:32.786336 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:32.786295    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:32.902263 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:32.902161    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"]
Oct 30 22:04:32.906329 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:32.906300    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"
Oct 30 22:04:32.957340 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:32.957299    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f48521-b5bd-4e36-8200-7129da431734-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z\" (UID: \"59f48521-b5bd-4e36-8200-7129da431734\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"
Oct 30 22:04:32.957553 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:32.957353    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmnrl\" (UniqueName: \"kubernetes.io/projected/59f48521-b5bd-4e36-8200-7129da431734-kube-api-access-wmnrl\") pod \"payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z\" (UID: \"59f48521-b5bd-4e36-8200-7129da431734\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"
Oct 30 22:04:32.961028 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:32.960937    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-9f8e92\""
Oct 30 22:04:33.013098 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.013060    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-9f8e92-dockercfg-65zl8\""
Oct 30 22:04:33.018772 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.018738    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"]
Oct 30 22:04:33.059013 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.058972    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f48521-b5bd-4e36-8200-7129da431734-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z\" (UID: \"59f48521-b5bd-4e36-8200-7129da431734\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"
Oct 30 22:04:33.059244 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.059039    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmnrl\" (UniqueName: \"kubernetes.io/projected/59f48521-b5bd-4e36-8200-7129da431734-kube-api-access-wmnrl\") pod \"payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z\" (UID: \"59f48521-b5bd-4e36-8200-7129da431734\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"
Oct 30 22:04:33.060222 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.060195    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f48521-b5bd-4e36-8200-7129da431734-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z\" (UID: \"59f48521-b5bd-4e36-8200-7129da431734\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"
Oct 30 22:04:33.074046 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.074008    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"]
Oct 30 22:04:33.078486 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.078444    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"
Oct 30 22:04:33.116746 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.116717    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmnrl\" (UniqueName: \"kubernetes.io/projected/59f48521-b5bd-4e36-8200-7129da431734-kube-api-access-wmnrl\") pod \"payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z\" (UID: \"59f48521-b5bd-4e36-8200-7129da431734\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"
Oct 30 22:04:33.140935 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.128555    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"]
Oct 30 22:04:33.170428 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.170390    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz4rw\" (UniqueName: \"kubernetes.io/projected/61e86a1d-3e10-4207-a9a1-d7a72072684d-kube-api-access-kz4rw\") pod \"payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt\" (UID: \"61e86a1d-3e10-4207-a9a1-d7a72072684d\") " pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"
Oct 30 22:04:33.170825 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.170623    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61e86a1d-3e10-4207-a9a1-d7a72072684d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt\" (UID: \"61e86a1d-3e10-4207-a9a1-d7a72072684d\") " pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"
Oct 30 22:04:33.223047 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.223010    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"
Oct 30 22:04:33.273311 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.273270    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61e86a1d-3e10-4207-a9a1-d7a72072684d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt\" (UID: \"61e86a1d-3e10-4207-a9a1-d7a72072684d\") " pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"
Oct 30 22:04:33.273511 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.273490    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz4rw\" (UniqueName: \"kubernetes.io/projected/61e86a1d-3e10-4207-a9a1-d7a72072684d-kube-api-access-kz4rw\") pod \"payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt\" (UID: \"61e86a1d-3e10-4207-a9a1-d7a72072684d\") " pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"
Oct 30 22:04:33.274089 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.274067    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61e86a1d-3e10-4207-a9a1-d7a72072684d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt\" (UID: \"61e86a1d-3e10-4207-a9a1-d7a72072684d\") " pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"
Oct 30 22:04:33.330385 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.330348    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz4rw\" (UniqueName: \"kubernetes.io/projected/61e86a1d-3e10-4207-a9a1-d7a72072684d-kube-api-access-kz4rw\") pod \"payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt\" (UID: \"61e86a1d-3e10-4207-a9a1-d7a72072684d\") " pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"
Oct 30 22:04:33.395469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.395430    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"
Oct 30 22:04:33.609413 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.609378    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"]
Oct 30 22:04:33.610634 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:04:33.610601    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59f48521_b5bd_4e36_8200_7129da431734.slice/crio-465d35141bd52cefee3970eadc5fa7e3f6efd25abc34c8b0b115184cdf227ebf WatchSource:0}: Error finding container 465d35141bd52cefee3970eadc5fa7e3f6efd25abc34c8b0b115184cdf227ebf: Status 404 returned error can't find the container with id 465d35141bd52cefee3970eadc5fa7e3f6efd25abc34c8b0b115184cdf227ebf
Oct 30 22:04:33.789335 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:33.789303    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"]
Oct 30 22:04:33.790940 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:04:33.790909    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61e86a1d_3e10_4207_a9a1_d7a72072684d.slice/crio-255b2872b2251df35f90d62c03a2e9b02d1f16c050354639c877d38788775d96 WatchSource:0}: Error finding container 255b2872b2251df35f90d62c03a2e9b02d1f16c050354639c877d38788775d96: Status 404 returned error can't find the container with id 255b2872b2251df35f90d62c03a2e9b02d1f16c050354639c877d38788775d96
Oct 30 22:04:34.186510 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:34.186469    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z" event={"ID":"59f48521-b5bd-4e36-8200-7129da431734","Type":"ContainerStarted","Data":"36ac064df86a2810764f02c3de1a731cd80da176d22a1701dc50cdbbbae94499"}
Oct 30 22:04:34.186742 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:34.186578    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"
Oct 30 22:04:34.186742 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:34.186609    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z" event={"ID":"59f48521-b5bd-4e36-8200-7129da431734","Type":"ContainerStarted","Data":"465d35141bd52cefee3970eadc5fa7e3f6efd25abc34c8b0b115184cdf227ebf"}
Oct 30 22:04:34.188355 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:34.188323    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt" event={"ID":"61e86a1d-3e10-4207-a9a1-d7a72072684d","Type":"ContainerStarted","Data":"63771e290ed92cb7553b0faa85456a09930449c85a16dee30bd9c40fb83f330e"}
Oct 30 22:04:34.188497 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:34.188377    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt" event={"ID":"61e86a1d-3e10-4207-a9a1-d7a72072684d","Type":"ContainerStarted","Data":"255b2872b2251df35f90d62c03a2e9b02d1f16c050354639c877d38788775d96"}
Oct 30 22:04:34.188497 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:34.188421    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"
Oct 30 22:04:34.302810 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:34.302743    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z" podStartSLOduration=2.302721961 podStartE2EDuration="2.302721961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:34.300592556 +0000 UTC m=+1816.972496492" watchObservedRunningTime="2026-10-30 22:04:34.302721961 +0000 UTC m=+1816.974625943"
Oct 30 22:04:34.424740 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:34.424669    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt" podStartSLOduration=1.424652552 podStartE2EDuration="1.424652552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:34.417009797 +0000 UTC m=+1817.088913758" watchObservedRunningTime="2026-10-30 22:04:34.424652552 +0000 UTC m=+1817.096556514"
Oct 30 22:04:35.533799 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:35.533759    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:41.169074 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:41.169019    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:42.812992 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:42.812957    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:43.268309 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.268269    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:43.773770 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.773732    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"]
Oct 30 22:04:43.777347 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.777329    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:43.950304 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.929486    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w0-485d1e-openshift-default-dockercfg-qnlqh\""
Oct 30 22:04:43.981030 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.980083    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:43.981030 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.980387    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"]
Oct 30 22:04:43.982745 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.982712    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a144510-a70d-4553-8717-b048d64094b3-istio-podinfo\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:43.982972 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.982954    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-workload-socket\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:43.983348 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.983326    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-workload-certs\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:43.983524 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.983505    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a144510-a70d-4553-8717-b048d64094b3-istio-token\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:43.983854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.983827    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a144510-a70d-4553-8717-b048d64094b3-istiod-ca-cert\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:43.983934 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.983901    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-istio-data\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:43.984397 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.984373    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8cwx\" (UniqueName: \"kubernetes.io/projected/6a144510-a70d-4553-8717-b048d64094b3-kube-api-access-x8cwx\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:43.984517 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.984439    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-credential-socket\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:43.985145 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:43.985118    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-istio-envoy\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.086924 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.086828    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-istio-envoy\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.086924 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.086890    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a144510-a70d-4553-8717-b048d64094b3-istio-podinfo\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.086924 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.086914    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-workload-socket\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.087380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.086955    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-workload-certs\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.087380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.087085    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a144510-a70d-4553-8717-b048d64094b3-istio-token\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.087380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.087205    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a144510-a70d-4553-8717-b048d64094b3-istiod-ca-cert\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.087380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.087291    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-istio-data\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.087587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.087395    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8cwx\" (UniqueName: \"kubernetes.io/projected/6a144510-a70d-4553-8717-b048d64094b3-kube-api-access-x8cwx\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.087587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.087405    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-workload-certs\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.087587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.087426    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-credential-socket\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.087587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.087434    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-workload-socket\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.087797 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.087638    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-istio-data\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.087860 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.087789    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-credential-socket\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.088012 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.087989    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a144510-a70d-4553-8717-b048d64094b3-istiod-ca-cert\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.089557 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.089529    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-istio-envoy\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.089833 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.089812    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a144510-a70d-4553-8717-b048d64094b3-istio-podinfo\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.155458 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.155422    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8cwx\" (UniqueName: \"kubernetes.io/projected/6a144510-a70d-4553-8717-b048d64094b3-kube-api-access-x8cwx\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.156573 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.156548    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a144510-a70d-4553-8717-b048d64094b3-istio-token\") pod \"e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") " pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.288296 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.288254    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:44.400749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.400658    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:44.877721 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.877685    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"]
Oct 30 22:04:44.909355 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:04:44.909316    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a144510_a70d_4553_8717_b048d64094b3.slice/crio-2ba8007ba53378b9c0ef2d1407eb70d6092543eb22eb758c10fb713200b7d58d WatchSource:0}: Error finding container 2ba8007ba53378b9c0ef2d1407eb70d6092543eb22eb758c10fb713200b7d58d: Status 404 returned error can't find the container with id 2ba8007ba53378b9c0ef2d1407eb70d6092543eb22eb758c10fb713200b7d58d
Oct 30 22:04:44.912115 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.912079    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:44.912286 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.912153    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:44.912286 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:44.912186    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:45.200550 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:45.200515    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"
Oct 30 22:04:45.200998 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:45.200610    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"
Oct 30 22:04:45.303815 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:45.303781    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz" event={"ID":"6a144510-a70d-4553-8717-b048d64094b3","Type":"ContainerStarted","Data":"b842587bdde38aae6135e1d4d5757ec25c2c859789ed08dbf1b215d1390f49d3"}
Oct 30 22:04:45.303994 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:45.303844    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz" event={"ID":"6a144510-a70d-4553-8717-b048d64094b3","Type":"ContainerStarted","Data":"2ba8007ba53378b9c0ef2d1407eb70d6092543eb22eb758c10fb713200b7d58d"}
Oct 30 22:04:45.401057 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:45.401018    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:45.586009 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:45.585889    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz" podStartSLOduration=2.585873651 podStartE2EDuration="2.585873651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:45.581814325 +0000 UTC m=+1828.253718261" watchObservedRunningTime="2026-10-30 22:04:45.585873651 +0000 UTC m=+1828.257777585"
Oct 30 22:04:46.407057 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:46.407025    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:46.548961 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:46.548915    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:47.318374 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:47.318340    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:47.319457 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:47.319438    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:04:48.362913 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:04:48.362877    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:04:48.470432 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:48.470386    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:48.682527 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:48.682492    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:51.012394 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:51.012357    5586 patch_prober.go:29] interesting pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:04:51.012916 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:51.012436    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:51.703708 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:51.703611    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:51.915310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:51.915273    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:52.363707 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:52.363616    5586 generic.go:356] "Generic (PLEG): container finished" podID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerID="190677165c88e636d400ced972849c243e1c36f772db8fde1702cb9a3caa88a7" exitCode=0
Oct 30 22:04:52.363707 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:52.363657    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" event={"ID":"e1307e68-d9b4-48e0-bad6-44854d56a0c4","Type":"ContainerDied","Data":"190677165c88e636d400ced972849c243e1c36f772db8fde1702cb9a3caa88a7"}
Oct 30 22:04:52.364128 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:52.364112    5586 scope.go:122] "RemoveContainer" containerID="190677165c88e636d400ced972849c243e1c36f772db8fde1702cb9a3caa88a7"
Oct 30 22:04:52.366107 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:52.366070    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:52.366256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:52.366167    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:52.366256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:52.366217    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:04:53.373412 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:53.373375    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" event={"ID":"e1307e68-d9b4-48e0-bad6-44854d56a0c4","Type":"ContainerStarted","Data":"00b90ec64c2d470a33e2c529e974894b70a8791c1f24ad975b7138c7dc5bd1b9"}
Oct 30 22:04:53.961204 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:53.961158    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:04:53.961543 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:53.961520    5586 patch_prober.go:29] interesting pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 22:04:53.961592 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:53.961567    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 22:04:54.961373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:54.961331    5586 patch_prober.go:29] interesting pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 22:04:54.961879 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:54.961412    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 22:04:55.961467 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:55.961426    5586 patch_prober.go:29] interesting pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 22:04:55.961906 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:55.961485    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 22:04:56.961839 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:56.961799    5586 patch_prober.go:29] interesting pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 22:04:56.962310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:56.961887    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 22:04:57.361702 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:57.360706    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:04:57.961956 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:57.961917    5586 patch_prober.go:29] interesting pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 22:04:57.962436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:57.961974    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 22:04:58.961360 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:58.961324    5586 patch_prober.go:29] interesting pod/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 22:04:58.961579 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:58.961424    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 22:04:59.966177 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:59.966137    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:04:59.966630 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:59.966535    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:04:59.967185 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:04:59.967166    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:05:02.803531 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:02.803481    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:03.373950 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:03.373899    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:07.097484 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.097449    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"]
Oct 30 22:05:07.101410 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.101390    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.155709 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.155684    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6df3ba0e\"/\"openshift-service-ca.crt\""
Oct 30 22:05:07.183397 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.183365    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92f71b18-38f6-4abe-ad38-afd53cbadeea-tls-certs\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.183557 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.183516    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjmsd\" (UniqueName: \"kubernetes.io/projected/92f71b18-38f6-4abe-ad38-afd53cbadeea-kube-api-access-qjmsd\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.183679 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.183662    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-model-cache\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.183741 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.183716    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-tmp-dir\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.183817 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.183791    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-dshm\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.183876 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.183844    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-home\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.203313 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.203286    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6df3ba0e\"/\"kube-root-ca.crt\""
Oct 30 22:05:07.203552 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.203537    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6df3ba0e\"/\"default-dockercfg-8wvhf\""
Oct 30 22:05:07.203610 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.203539    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6df3ba0e\"/\"e2e-lifecycle-model-6df3ba0e-kserve-self-signed-certs\""
Oct 30 22:05:07.209428 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.209400    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"]
Oct 30 22:05:07.213001 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.212974    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:07.284891 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.284858    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-tmp-dir\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.285087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.284901    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-dshm\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.285087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.284923    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-home\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.285087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.284972    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92f71b18-38f6-4abe-ad38-afd53cbadeea-tls-certs\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.285087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.285077    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjmsd\" (UniqueName: \"kubernetes.io/projected/92f71b18-38f6-4abe-ad38-afd53cbadeea-kube-api-access-qjmsd\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.285299 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.285221    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-model-cache\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.285522 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.285501    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-home\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.285628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.285538    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-tmp-dir\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.285628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.285579    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-model-cache\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.287389 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.287367    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-dshm\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.287653 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.287635    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92f71b18-38f6-4abe-ad38-afd53cbadeea-tls-certs\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.342677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.342635    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjmsd\" (UniqueName: \"kubernetes.io/projected/92f71b18-38f6-4abe-ad38-afd53cbadeea-kube-api-access-qjmsd\") pod \"e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") " pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.413532 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.413494    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:07.793942 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:07.793911    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"]
Oct 30 22:05:07.794904 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:05:07.794875    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92f71b18_38f6_4abe_ad38_afd53cbadeea.slice/crio-f11ce377cd7f2f42a7d032e849f452a4f25269c3a6718a778df143bcf160ae56 WatchSource:0}: Error finding container f11ce377cd7f2f42a7d032e849f452a4f25269c3a6718a778df143bcf160ae56: Status 404 returned error can't find the container with id f11ce377cd7f2f42a7d032e849f452a4f25269c3a6718a778df143bcf160ae56
Oct 30 22:05:08.121960 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:08.121863    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:08.490733 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:08.490694    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt" event={"ID":"92f71b18-38f6-4abe-ad38-afd53cbadeea","Type":"ContainerStarted","Data":"1446bae01d56a7471668a82c71152ffcb90a1d3f3019b2122457650e88df998c"}
Oct 30 22:05:08.490956 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:08.490746    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt" event={"ID":"92f71b18-38f6-4abe-ad38-afd53cbadeea","Type":"ContainerStarted","Data":"f11ce377cd7f2f42a7d032e849f452a4f25269c3a6718a778df143bcf160ae56"}
Oct 30 22:05:08.609420 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:08.609346    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt" podStartSLOduration=1.6093277609999999 podStartE2EDuration="1.609327761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:08.608948057 +0000 UTC m=+1851.280851993" watchObservedRunningTime="2026-10-30 22:05:08.609327761 +0000 UTC m=+1851.281231696"
Oct 30 22:05:09.414323 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:09.414289    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:13.483860 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:13.483816    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:17.414192 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:17.414149    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:17.427030 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:17.427000    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:17.585421 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:17.585385    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:05:18.342420 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.342385    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"]
Oct 30 22:05:18.347130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.347100    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:18.365542 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:05:18.365507    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:05:18.386459 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.386425    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:18.386610 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.386475    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:18.386740 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.386701    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:18.386960 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.386940    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fq2d\" (UniqueName: \"kubernetes.io/projected/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-kube-api-access-4fq2d\") pod \"maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:18.404460 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.404422    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"]
Oct 30 22:05:18.448159 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.448123    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-20c7c7-serving-cert\""
Oct 30 22:05:18.488097 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.488055    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fq2d\" (UniqueName: \"kubernetes.io/projected/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-kube-api-access-4fq2d\") pod \"maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:18.488363 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.488172    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:18.488363 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.488209    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:18.488489 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.488384    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:18.488642 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:05:18.488605    5586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-serving-cert: secret "maas-api-e2e-shared-b-w3-20c7c7-serving-cert" not found
Oct 30 22:05:18.488788 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:05:18.488693    5586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-maas-api-tls podName:377ab9b6-ae8c-4acd-a7d7-9a8426031b26 nodeName:}" failed. No retries permitted until 2026-10-30 22:05:18.988670485 +0000 UTC m=+1861.660574433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-maas-api-tls") pod "maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8" (UID: "377ab9b6-ae8c-4acd-a7d7-9a8426031b26") : secret "maas-api-e2e-shared-b-w3-20c7c7-serving-cert" not found
Oct 30 22:05:18.489117 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.489093    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:18.491111 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.491086    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:18.548070 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.548027    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fq2d\" (UniqueName: \"kubernetes.io/projected/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-kube-api-access-4fq2d\") pod \"maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:18.584857 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.584817    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:18.885041 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.885002    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:18.995119 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.995078    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:18.997760 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:18.997734    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:19.105769 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:19.105728    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:19.263819 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:19.263775    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:19.651785 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:19.651758    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"]
Oct 30 22:05:19.652972 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:05:19.652936    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod377ab9b6_ae8c_4acd_a7d7_9a8426031b26.slice/crio-10dc9511052b1fd402ffc17036fd89a7f2d0d16e98a78b27ba876b3a37b20bc0 WatchSource:0}: Error finding container 10dc9511052b1fd402ffc17036fd89a7f2d0d16e98a78b27ba876b3a37b20bc0: Status 404 returned error can't find the container with id 10dc9511052b1fd402ffc17036fd89a7f2d0d16e98a78b27ba876b3a37b20bc0
Oct 30 22:05:20.600102 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:20.600063    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8" event={"ID":"377ab9b6-ae8c-4acd-a7d7-9a8426031b26","Type":"ContainerStarted","Data":"7a5396c5703950f50b8ea14e3895ef5cd48ae3ed3a68cc3e1bf889b9c95f2849"}
Oct 30 22:05:20.600301 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:20.600132    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:20.600301 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:20.600150    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8" event={"ID":"377ab9b6-ae8c-4acd-a7d7-9a8426031b26","Type":"ContainerStarted","Data":"10dc9511052b1fd402ffc17036fd89a7f2d0d16e98a78b27ba876b3a37b20bc0"}
Oct 30 22:05:20.735217 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:20.735162    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8" podStartSLOduration=1.9015303700000001 podStartE2EDuration="2.735145273s" totalImagesPullingTime="833.614903ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:20.730941793 +0000 UTC m=+1863.402845738" watchObservedRunningTime="2026-10-30 22:05:20.735145273 +0000 UTC m=+1863.407049209"
Oct 30 22:05:23.687199 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:23.687168    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:24.702692 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:24.702631    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:24.862285 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:24.861469    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:26.215703 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:26.215649    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:26.422529 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:26.422473    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:26.613914 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:26.613841    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:05:28.588464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:28.588430    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:33.594639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:33.594599    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:35.123136 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:35.123097    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:35.585553 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:35.585513    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:36.541154 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:36.541114    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:37.435565 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:37.435518    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:38.803775 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:38.803729    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:44.117510 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:44.117473    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:46.117845 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.117802    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"]
Oct 30 22:05:46.127084 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.127054    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.189673 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.189640    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-6cd200-openshift-default-dockercfg-2vn78\""
Oct 30 22:05:46.214863 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.212614    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:46.221921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.221083    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2sx9\" (UniqueName: \"kubernetes.io/projected/89b841c7-c736-4ffe-b31e-762908d62361-kube-api-access-s2sx9\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.221921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.221292    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-istio-data\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.221921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.221347    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89b841c7-c736-4ffe-b31e-762908d62361-istio-podinfo\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.221921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.221402    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-istio-envoy\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.221921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.221440    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89b841c7-c736-4ffe-b31e-762908d62361-istiod-ca-cert\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.221921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.221475    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-workload-socket\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.221921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.221562    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-workload-certs\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.221921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.221624    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-credential-socket\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.221921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.221668    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89b841c7-c736-4ffe-b31e-762908d62361-istio-token\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.245659 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.245626    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"]
Oct 30 22:05:46.322733 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.322699    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-credential-socket\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.322917 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.322746    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89b841c7-c736-4ffe-b31e-762908d62361-istio-token\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.322917 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.322795    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2sx9\" (UniqueName: \"kubernetes.io/projected/89b841c7-c736-4ffe-b31e-762908d62361-kube-api-access-s2sx9\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.322917 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.322890    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-istio-data\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.323049 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.322933    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89b841c7-c736-4ffe-b31e-762908d62361-istio-podinfo\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.323049 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.322988    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-istio-envoy\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.323049 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.323025    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89b841c7-c736-4ffe-b31e-762908d62361-istiod-ca-cert\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.323274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.323061    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-workload-socket\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.323274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.323085    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-credential-socket\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.323274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.323155    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-workload-certs\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.323525 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.323333    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-istio-data\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.323525 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.323456    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-workload-socket\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.323607 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.323525    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-workload-certs\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.323754 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.323726    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89b841c7-c736-4ffe-b31e-762908d62361-istiod-ca-cert\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.325403 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.325377    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-istio-envoy\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.325675 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.325656    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89b841c7-c736-4ffe-b31e-762908d62361-istio-podinfo\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.369421 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.369325    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:46.380088 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.380056    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89b841c7-c736-4ffe-b31e-762908d62361-istio-token\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.380088 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.380082    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2sx9\" (UniqueName: \"kubernetes.io/projected/89b841c7-c736-4ffe-b31e-762908d62361-kube-api-access-s2sx9\") pod \"e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") " pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.546026 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.545992    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:46.651285 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.647127    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:46.801542 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.801504    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:46.989078 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.989051    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"]
Oct 30 22:05:46.990578 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:05:46.990553    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89b841c7_c736_4ffe_b31e_762908d62361.slice/crio-dc4ed49844729e51aca3063cd9bd0e250555bcb1c951204b9f0abd4d612eb1d4 WatchSource:0}: Error finding container dc4ed49844729e51aca3063cd9bd0e250555bcb1c951204b9f0abd4d612eb1d4: Status 404 returned error can't find the container with id dc4ed49844729e51aca3063cd9bd0e250555bcb1c951204b9f0abd4d612eb1d4
Oct 30 22:05:46.993279 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.993223    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:46.993392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.993314    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:46.993392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:46.993345    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:47.460492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.460458    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"]
Oct 30 22:05:47.466182 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.466154    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.518436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.518402    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w6-1db768-openshift-default-dockercfg-p4tjf\""
Oct 30 22:05:47.560633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.560593    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-podinfo\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.560940 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.560905    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:47.560940 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.560924    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istiod-ca-cert\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.561120 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.560970    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-workload-certs\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.561175 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.561115    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-workload-socket\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.561274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.561184    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-token\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.561274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.561263    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-credential-socket\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.561394 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.561332    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwzp5\" (UniqueName: \"kubernetes.io/projected/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-kube-api-access-fwzp5\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.561394 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.561362    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-envoy\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.561487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.561408    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-data\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.573754 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.573716    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"]
Oct 30 22:05:47.663555 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.663517    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-workload-socket\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.663739 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.663569    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-token\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.663739 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.663613    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-credential-socket\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.663739 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.663660    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwzp5\" (UniqueName: \"kubernetes.io/projected/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-kube-api-access-fwzp5\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.663739 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.663679    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-envoy\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.663739 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.663709    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-data\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.663907 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.663840    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-podinfo\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.663907 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.663875    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istiod-ca-cert\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.663907 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.663899    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-workload-certs\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.664355 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.664326    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-workload-certs\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.664688 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.664661    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-workload-socket\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.664805 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.664785    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-credential-socket\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.664952 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.664929    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-data\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.665297 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.665272    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istiod-ca-cert\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.667227 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.667204    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-envoy\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.667519 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.667496    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-podinfo\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.739476 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.739398    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-token\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.739652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.739474    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwzp5\" (UniqueName: \"kubernetes.io/projected/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-kube-api-access-fwzp5\") pod \"e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") " pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.783499 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.783461    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:47.869059 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.869015    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h" event={"ID":"89b841c7-c736-4ffe-b31e-762908d62361","Type":"ContainerStarted","Data":"a410127d6bfb122a0f4875cd14b5d6737ebce0c77e381e0d91f7e61b551bca62"}
Oct 30 22:05:47.869282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.869086    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h" event={"ID":"89b841c7-c736-4ffe-b31e-762908d62361","Type":"ContainerStarted","Data":"dc4ed49844729e51aca3063cd9bd0e250555bcb1c951204b9f0abd4d612eb1d4"}
Oct 30 22:05:47.910101 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.906730    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:47.996709 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:47.996588    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h" podStartSLOduration=1.9965727580000001 podStartE2EDuration="1.996572758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:47.989984404 +0000 UTC m=+1890.661888341" watchObservedRunningTime="2026-10-30 22:05:47.996572758 +0000 UTC m=+1890.668476693"
Oct 30 22:05:48.275195 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:48.275167    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"]
Oct 30 22:05:48.276739 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:05:48.276716    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d056d9_bcc2_4af8_a50d_f46d2efd7eb7.slice/crio-f2d9979076f29f3ffc2d2219e13e996516270a5ad9cb228061458a07285e4f7d WatchSource:0}: Error finding container f2d9979076f29f3ffc2d2219e13e996516270a5ad9cb228061458a07285e4f7d: Status 404 returned error can't find the container with id f2d9979076f29f3ffc2d2219e13e996516270a5ad9cb228061458a07285e4f7d
Oct 30 22:05:48.279446 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:48.279417    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:48.279518 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:48.279485    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:48.279518 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:48.279515    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:05:48.362998 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:05:48.362959    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:05:48.546953 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:48.546862    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:48.554339 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:48.554313    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:48.879537 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:48.879436    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx" event={"ID":"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7","Type":"ContainerStarted","Data":"183b7260b4d2ee4c0c91cde2927605d1dbe4d7065288829b79369a7bc3ad52f6"}
Oct 30 22:05:48.879537 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:48.879490    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx" event={"ID":"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7","Type":"ContainerStarted","Data":"f2d9979076f29f3ffc2d2219e13e996516270a5ad9cb228061458a07285e4f7d"}
Oct 30 22:05:48.879813 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:48.879793    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:48.880758 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:48.880738    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:05:48.998420 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:48.998366    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx" podStartSLOduration=1.998353313 podStartE2EDuration="1.998353313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:48.995529894 +0000 UTC m=+1891.667433827" watchObservedRunningTime="2026-10-30 22:05:48.998353313 +0000 UTC m=+1891.670257247"
Oct 30 22:05:49.423631 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.423597    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"]
Oct 30 22:05:49.428058 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.428040    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:49.480335 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.480300    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-9f8e92-serving-cert\""
Oct 30 22:05:49.485260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.485213    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"]
Oct 30 22:05:49.492001 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.491969    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-maas-api-tls\") pod \"maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:49.492145 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.492101    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-metrics-certs\") pod \"maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:49.492245 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.492214    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5deff068-24f9-412b-a6ec-f3d106d24002-openshift-service-ca\") pod \"maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:49.492442 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.492423    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xl7d\" (UniqueName: \"kubernetes.io/projected/5deff068-24f9-412b-a6ec-f3d106d24002-kube-api-access-2xl7d\") pod \"maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:49.593603 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.593568    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-maas-api-tls\") pod \"maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:49.594007 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.593621    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-metrics-certs\") pod \"maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:49.594007 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.593670    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5deff068-24f9-412b-a6ec-f3d106d24002-openshift-service-ca\") pod \"maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:49.594007 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:05:49.593710    5586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-serving-cert: secret "maas-api-e2e-api-a-9f8e92-serving-cert" not found
Oct 30 22:05:49.594007 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:05:49.593767    5586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-maas-api-tls podName:5deff068-24f9-412b-a6ec-f3d106d24002 nodeName:}" failed. No retries permitted until 2026-10-30 22:05:50.093751837 +0000 UTC m=+1892.765655776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-maas-api-tls") pod "maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4" (UID: "5deff068-24f9-412b-a6ec-f3d106d24002") : secret "maas-api-e2e-api-a-9f8e92-serving-cert" not found
Oct 30 22:05:49.594007 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.593809    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xl7d\" (UniqueName: \"kubernetes.io/projected/5deff068-24f9-412b-a6ec-f3d106d24002-kube-api-access-2xl7d\") pod \"maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:49.594355 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.594335    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5deff068-24f9-412b-a6ec-f3d106d24002-openshift-service-ca\") pod \"maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:49.596117 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.596099    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-metrics-certs\") pod \"maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:49.610386 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.610354    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:49.661418 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.660020    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xl7d\" (UniqueName: \"kubernetes.io/projected/5deff068-24f9-412b-a6ec-f3d106d24002-kube-api-access-2xl7d\") pod \"maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:49.784431 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.784341    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:49.790559 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.790531    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:49.812567 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.812535    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:49.887569 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.887539    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:49.888454 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.888429    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:05:49.962502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:49.962456    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:50.103679 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:50.103587    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-maas-api-tls\") pod \"maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:50.106064 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:50.106033    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-maas-api-tls\") pod \"maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:50.343543 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:50.343502    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:50.723749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:50.723720    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"]
Oct 30 22:05:50.725759 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:05:50.725730    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5deff068_24f9_412b_a6ec_f3d106d24002.slice/crio-589760ad9b9e07fe3b8893f7ccde902504d105de3d855125914e8e2772121417 WatchSource:0}: Error finding container 589760ad9b9e07fe3b8893f7ccde902504d105de3d855125914e8e2772121417: Status 404 returned error can't find the container with id 589760ad9b9e07fe3b8893f7ccde902504d105de3d855125914e8e2772121417
Oct 30 22:05:50.896266 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:50.896192    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4" event={"ID":"5deff068-24f9-412b-a6ec-f3d106d24002","Type":"ContainerStarted","Data":"589760ad9b9e07fe3b8893f7ccde902504d105de3d855125914e8e2772121417"}
Oct 30 22:05:51.738428 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:51.738384    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:51.903091 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:51.902994    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:51.906890 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:51.906860    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4" event={"ID":"5deff068-24f9-412b-a6ec-f3d106d24002","Type":"ContainerStarted","Data":"6af4467d62c1b410e5e8d1ef4c7871535f4b7bca7016f6a546f0f2c596fcf3c7"}
Oct 30 22:05:52.021733 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:52.021676    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4" podStartSLOduration=2.148393628 podStartE2EDuration="3.021660712s" totalImagesPullingTime="873.267084ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:52.020436995 +0000 UTC m=+1894.692340943" watchObservedRunningTime="2026-10-30 22:05:52.021660712 +0000 UTC m=+1894.693564648"
Oct 30 22:05:52.064969 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:52.064936    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:52.914892 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:52.914864    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:05:54.801044 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:54.801000    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:55.005228 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:55.005191    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:55.251330 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:55.251288    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:56.672716 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:56.672680    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:56.861034 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:56.861001    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:05:58.934785 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:05:58.934691    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:06:00.165058 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:00.165009    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:00.872542 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:00.872503    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:01.082046 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:01.082011    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:01.240926 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:01.240888    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:05.047306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:05.047222    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:05.284854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:05.284813    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:05.456198 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:05.456150    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:09.987928 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:09.987886    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"]
Oct 30 22:06:09.988479 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:09.988142    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r" podUID="1baa2c91-37cd-4e6c-a85d-8674138b03fe" containerName="maas-api" containerID="cri-o://ffbbc35e9d21818a4d21b4c3ceece5d98fdabf59ffb8dc320ceef26dbdefaad6" gracePeriod=30
Oct 30 22:06:10.106198 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:10.106166    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:10.318366 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:10.318330    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:06:10.412079 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:10.412038    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1baa2c91-37cd-4e6c-a85d-8674138b03fe-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1baa2c91-37cd-4e6c-a85d-8674138b03fe-openshift-service-ca\") pod \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") "
Oct 30 22:06:10.412335 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:10.412315    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1baa2c91-37cd-4e6c-a85d-8674138b03fe-kube-api-access-nm48d\" (UniqueName: \"kubernetes.io/projected/1baa2c91-37cd-4e6c-a85d-8674138b03fe-kube-api-access-nm48d\") pod \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") "
Oct 30 22:06:10.412450 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:10.412433    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-metrics-certs\") pod \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") "
Oct 30 22:06:10.412534 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:10.412454    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1baa2c91-37cd-4e6c-a85d-8674138b03fe-openshift-service-ca" pod "1baa2c91-37cd-4e6c-a85d-8674138b03fe" (UID: "1baa2c91-37cd-4e6c-a85d-8674138b03fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:06:10.412595 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:10.412559    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-maas-api-tls\") pod \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\" (UID: \"1baa2c91-37cd-4e6c-a85d-8674138b03fe\") "
Oct 30 22:06:10.413248 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:10.413207    5586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1baa2c91-37cd-4e6c-a85d-8674138b03fe-openshift-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:10.416104 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:10.416067    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1baa2c91-37cd-4e6c-a85d-8674138b03fe-kube-api-access-nm48d" pod "1baa2c91-37cd-4e6c-a85d-8674138b03fe" (UID: "1baa2c91-37cd-4e6c-a85d-8674138b03fe"). InnerVolumeSpecName "kube-api-access-nm48d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:06:10.416291 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:10.416170    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-maas-api-tls" pod "1baa2c91-37cd-4e6c-a85d-8674138b03fe" (UID: "1baa2c91-37cd-4e6c-a85d-8674138b03fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:06:10.424060 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:10.424024    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-metrics-certs" pod "1baa2c91-37cd-4e6c-a85d-8674138b03fe" (UID: "1baa2c91-37cd-4e6c-a85d-8674138b03fe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:06:10.515087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:10.515042    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-maas-api-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:10.515087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:10.515077    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nm48d\" (UniqueName: \"kubernetes.io/projected/1baa2c91-37cd-4e6c-a85d-8674138b03fe-kube-api-access-nm48d\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:10.515337 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:10.515116    5586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1baa2c91-37cd-4e6c-a85d-8674138b03fe-metrics-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:11.128302 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.121448    5586 generic.go:356] "Generic (PLEG): container finished" podID="1baa2c91-37cd-4e6c-a85d-8674138b03fe" containerID="ffbbc35e9d21818a4d21b4c3ceece5d98fdabf59ffb8dc320ceef26dbdefaad6" exitCode=0
Oct 30 22:06:11.128302 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.121499    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r" event={"ID":"1baa2c91-37cd-4e6c-a85d-8674138b03fe","Type":"ContainerDied","Data":"ffbbc35e9d21818a4d21b4c3ceece5d98fdabf59ffb8dc320ceef26dbdefaad6"}
Oct 30 22:06:11.128302 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.121551    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r" event={"ID":"1baa2c91-37cd-4e6c-a85d-8674138b03fe","Type":"ContainerDied","Data":"5c524c40ebf10828fe72eb1f6cabbb3f93fea89f7e2131e28075fdd688a454de"}
Oct 30 22:06:11.128302 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.121580    5586 scope.go:122] "RemoveContainer" containerID="ffbbc35e9d21818a4d21b4c3ceece5d98fdabf59ffb8dc320ceef26dbdefaad6"
Oct 30 22:06:11.128302 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.121739    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"
Oct 30 22:06:11.148127 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.148103    5586 scope.go:122] "RemoveContainer" containerID="ffbbc35e9d21818a4d21b4c3ceece5d98fdabf59ffb8dc320ceef26dbdefaad6"
Oct 30 22:06:11.148674 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:06:11.148642    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffbbc35e9d21818a4d21b4c3ceece5d98fdabf59ffb8dc320ceef26dbdefaad6\": container with ID starting with ffbbc35e9d21818a4d21b4c3ceece5d98fdabf59ffb8dc320ceef26dbdefaad6 not found: ID does not exist" containerID="ffbbc35e9d21818a4d21b4c3ceece5d98fdabf59ffb8dc320ceef26dbdefaad6"
Oct 30 22:06:11.148800 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.148682    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffbbc35e9d21818a4d21b4c3ceece5d98fdabf59ffb8dc320ceef26dbdefaad6"} err="failed to get container status \"ffbbc35e9d21818a4d21b4c3ceece5d98fdabf59ffb8dc320ceef26dbdefaad6\": rpc error: code = NotFound desc = could not find container \"ffbbc35e9d21818a4d21b4c3ceece5d98fdabf59ffb8dc320ceef26dbdefaad6\": container with ID starting with ffbbc35e9d21818a4d21b4c3ceece5d98fdabf59ffb8dc320ceef26dbdefaad6 not found: ID does not exist"
Oct 30 22:06:11.297765 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.297724    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"]
Oct 30 22:06:11.346008 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.345960    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e2deec91-f876f9c76-kbs8r"]
Oct 30 22:06:11.453407 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.453365    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1baa2c91-37cd-4e6c-a85d-8674138b03fe" path="/var/lib/kubelet/pods/1baa2c91-37cd-4e6c-a85d-8674138b03fe/volumes"
Oct 30 22:06:11.692827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.692787    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"]
Oct 30 22:06:11.693639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.693622    5586 cpu_manager.go:414] "containerMap: removing container" podUID="1baa2c91-37cd-4e6c-a85d-8674138b03fe" containerName="maas-api"
Oct 30 22:06:11.693688 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.693641    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1baa2c91-37cd-4e6c-a85d-8674138b03fe" containerName="maas-api"
Oct 30 22:06:11.693798 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.693769    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="1baa2c91-37cd-4e6c-a85d-8674138b03fe" containerName="maas-api"
Oct 30 22:06:11.699299 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.699267    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:11.758709 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.758669    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:11.758899 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.758843    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:11.758957 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.758921    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ec046a-92b3-4116-a7ed-dde35559ccaa-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:11.759016 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.758995    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sn88\" (UniqueName: \"kubernetes.io/projected/d6ec046a-92b3-4116-a7ed-dde35559ccaa-kube-api-access-7sn88\") pod \"maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:11.766049 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.766009    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"]
Oct 30 22:06:11.766298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.766177    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-485d1e-serving-cert\""
Oct 30 22:06:11.774718 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.774415    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:11.860961 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.860921    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sn88\" (UniqueName: \"kubernetes.io/projected/d6ec046a-92b3-4116-a7ed-dde35559ccaa-kube-api-access-7sn88\") pod \"maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:11.861138 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.861104    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:11.861342 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.861322    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:11.861431 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.861414    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ec046a-92b3-4116-a7ed-dde35559ccaa-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:11.861475 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:06:11.861443    5586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-serving-cert: secret "maas-api-e2e-shared-b-w0-485d1e-serving-cert" not found
Oct 30 22:06:11.861516 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:06:11.861507    5586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-maas-api-tls podName:d6ec046a-92b3-4116-a7ed-dde35559ccaa nodeName:}" failed. No retries permitted until 2026-10-30 22:06:12.361486968 +0000 UTC m=+1915.033390900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-maas-api-tls") pod "maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2" (UID: "d6ec046a-92b3-4116-a7ed-dde35559ccaa") : secret "maas-api-e2e-shared-b-w0-485d1e-serving-cert" not found
Oct 30 22:06:11.862111 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.862092    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ec046a-92b3-4116-a7ed-dde35559ccaa-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:11.863754 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.863733    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:11.921306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.921265    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sn88\" (UniqueName: \"kubernetes.io/projected/d6ec046a-92b3-4116-a7ed-dde35559ccaa-kube-api-access-7sn88\") pod \"maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:11.980843 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:11.980738    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:12.164409 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:12.162399    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:12.369782 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:12.369696    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:12.372342 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:12.372317    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:12.615402 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:12.615353    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:12.693015 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:12.692975    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:13.023631 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:13.023577    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"]
Oct 30 22:06:13.024820 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:06:13.024789    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6ec046a_92b3_4116_a7ed_dde35559ccaa.slice/crio-9e39b5dde440192fb4c2d371a9b4a00b3daf56ef166debb079a0cce9f1aad465 WatchSource:0}: Error finding container 9e39b5dde440192fb4c2d371a9b4a00b3daf56ef166debb079a0cce9f1aad465: Status 404 returned error can't find the container with id 9e39b5dde440192fb4c2d371a9b4a00b3daf56ef166debb079a0cce9f1aad465
Oct 30 22:06:13.184896 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:13.184852    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2" event={"ID":"d6ec046a-92b3-4116-a7ed-dde35559ccaa","Type":"ContainerStarted","Data":"9e39b5dde440192fb4c2d371a9b4a00b3daf56ef166debb079a0cce9f1aad465"}
Oct 30 22:06:14.197447 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:14.197402    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2" event={"ID":"d6ec046a-92b3-4116-a7ed-dde35559ccaa","Type":"ContainerStarted","Data":"dd0cb42801d7f3c555f412c60b84b295988208144d71b78885b97591c49996c4"}
Oct 30 22:06:14.198602 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:14.197616    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:14.315896 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:14.315840    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2" podStartSLOduration=2.495968168 podStartE2EDuration="3.31582295s" totalImagesPullingTime="819.854782ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:06:14.313538725 +0000 UTC m=+1916.985442660" watchObservedRunningTime="2026-10-30 22:06:14.31582295 +0000 UTC m=+1916.987726885"
Oct 30 22:06:15.207129 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:15.207096    5586 generic.go:356] "Generic (PLEG): container finished" podID="c0960ade-ab00-437c-84a2-2c9e702a9d06" containerID="ff9499ce9242c4dc1ef67356bbfaef27d506b1be8433c43b4cdf3fef4a87332c" exitCode=0
Oct 30 22:06:15.207633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:15.207166    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7" event={"ID":"c0960ade-ab00-437c-84a2-2c9e702a9d06","Type":"ContainerDied","Data":"ff9499ce9242c4dc1ef67356bbfaef27d506b1be8433c43b4cdf3fef4a87332c"}
Oct 30 22:06:15.207726 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:15.207658    5586 scope.go:122] "RemoveContainer" containerID="ff9499ce9242c4dc1ef67356bbfaef27d506b1be8433c43b4cdf3fef4a87332c"
Oct 30 22:06:15.209481 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:15.209455    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:06:15.209585 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:15.209520    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:06:15.209585 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:15.209552    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:06:16.259956 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:16.259910    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7" event={"ID":"c0960ade-ab00-437c-84a2-2c9e702a9d06","Type":"ContainerStarted","Data":"c4b6289a1b01df413d1d74a5283cd64349cc9c78e0fcb799cafa7cf26622b42d"}
Oct 30 22:06:16.581545 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:16.581448    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:06:16.583130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:16.583099    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:06:16.583274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:16.583161    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7" podUID="c0960ade-ab00-437c-84a2-2c9e702a9d06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 22:06:17.397880 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:17.397841    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:17.581512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:17.581471    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:06:17.581710 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:17.581542    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7" podUID="c0960ade-ab00-437c-84a2-2c9e702a9d06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 22:06:17.665587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:17.665534    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:06:18.389437 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:06:18.389367    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:06:18.581584 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:18.581539    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:06:18.582015 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:18.581602    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7" podUID="c0960ade-ab00-437c-84a2-2c9e702a9d06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 22:06:19.488932 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:19.488892    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:19.581955 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:19.581918    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:06:19.582492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:19.581985    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7" podUID="c0960ade-ab00-437c-84a2-2c9e702a9d06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 22:06:19.658222 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:19.658185    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:19.810976 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:19.810892    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"]
Oct 30 22:06:19.811144 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:19.811122    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp" podUID="08f613b9-8231-4e10-8a70-5dee789fc9f7" containerName="istio-proxy" containerID="cri-o://15c3e146d1d909f20a7aa68356811347214756397e7946f7cc4a656850c1f6b6" gracePeriod=30
Oct 30 22:06:19.837259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:19.835736    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:20.214487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:20.214461    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:06:20.582034 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:20.581939    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:06:20.582034 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:20.582020    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7" podUID="c0960ade-ab00-437c-84a2-2c9e702a9d06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 30 22:06:21.148117 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:21.148067    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:21.807221 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:21.807184    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:06:21.811559 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:21.811517    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:06:22.535906 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:22.535870    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:22.746391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:22.746348    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:22.947056 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:22.947018    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:25.106432 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.106400    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:06:25.237765 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.237725    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/08f613b9-8231-4e10-8a70-5dee789fc9f7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08f613b9-8231-4e10-8a70-5dee789fc9f7-istiod-ca-cert\") pod \"08f613b9-8231-4e10-8a70-5dee789fc9f7\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") "
Oct 30 22:06:25.237979 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.237797    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-envoy\") pod \"08f613b9-8231-4e10-8a70-5dee789fc9f7\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") "
Oct 30 22:06:25.237979 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.237820    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-workload-socket\") pod \"08f613b9-8231-4e10-8a70-5dee789fc9f7\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") "
Oct 30 22:06:25.237979 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.237856    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-podinfo\") pod \"08f613b9-8231-4e10-8a70-5dee789fc9f7\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") "
Oct 30 22:06:25.237979 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.237958    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08f613b9-8231-4e10-8a70-5dee789fc9f7-kube-api-access-hn26f\" (UniqueName: \"kubernetes.io/projected/08f613b9-8231-4e10-8a70-5dee789fc9f7-kube-api-access-hn26f\") pod \"08f613b9-8231-4e10-8a70-5dee789fc9f7\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") "
Oct 30 22:06:25.238209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.238057    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-data\") pod \"08f613b9-8231-4e10-8a70-5dee789fc9f7\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") "
Oct 30 22:06:25.238209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.238166    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-workload-certs\") pod \"08f613b9-8231-4e10-8a70-5dee789fc9f7\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") "
Oct 30 22:06:25.238209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.238198    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-credential-socket\") pod \"08f613b9-8231-4e10-8a70-5dee789fc9f7\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") "
Oct 30 22:06:25.238439 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.238196    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-workload-socket" pod "08f613b9-8231-4e10-8a70-5dee789fc9f7" (UID: "08f613b9-8231-4e10-8a70-5dee789fc9f7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:06:25.238439 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.238206    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08f613b9-8231-4e10-8a70-5dee789fc9f7-istiod-ca-cert" pod "08f613b9-8231-4e10-8a70-5dee789fc9f7" (UID: "08f613b9-8231-4e10-8a70-5dee789fc9f7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:06:25.238439 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.238406    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-token\" (UniqueName: \"kubernetes.io/projected/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-token\") pod \"08f613b9-8231-4e10-8a70-5dee789fc9f7\" (UID: \"08f613b9-8231-4e10-8a70-5dee789fc9f7\") "
Oct 30 22:06:25.238807 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.238500    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-data" pod "08f613b9-8231-4e10-8a70-5dee789fc9f7" (UID: "08f613b9-8231-4e10-8a70-5dee789fc9f7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:06:25.238807 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.238540    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-workload-certs" pod "08f613b9-8231-4e10-8a70-5dee789fc9f7" (UID: "08f613b9-8231-4e10-8a70-5dee789fc9f7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:06:25.238807 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.238568    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-credential-socket" pod "08f613b9-8231-4e10-8a70-5dee789fc9f7" (UID: "08f613b9-8231-4e10-8a70-5dee789fc9f7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:06:25.239122 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.239102    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:25.239193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.239128    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:25.239193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.239144    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08f613b9-8231-4e10-8a70-5dee789fc9f7-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:25.239193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.239158    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:25.239193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.239172    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:25.240417 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.240389    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-envoy" pod "08f613b9-8231-4e10-8a70-5dee789fc9f7" (UID: "08f613b9-8231-4e10-8a70-5dee789fc9f7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:06:25.240765 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.240733    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08f613b9-8231-4e10-8a70-5dee789fc9f7-kube-api-access-hn26f" pod "08f613b9-8231-4e10-8a70-5dee789fc9f7" (UID: "08f613b9-8231-4e10-8a70-5dee789fc9f7"). InnerVolumeSpecName "kube-api-access-hn26f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:06:25.240765 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.240744    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-podinfo" pod "08f613b9-8231-4e10-8a70-5dee789fc9f7" (UID: "08f613b9-8231-4e10-8a70-5dee789fc9f7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:06:25.240898 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.240806    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-token" pod "08f613b9-8231-4e10-8a70-5dee789fc9f7" (UID: "08f613b9-8231-4e10-8a70-5dee789fc9f7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:06:25.340535 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.340455    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:25.340535 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.340487    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:25.340535 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.340498    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn26f\" (UniqueName: \"kubernetes.io/projected/08f613b9-8231-4e10-8a70-5dee789fc9f7-kube-api-access-hn26f\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:25.340535 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.340508    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08f613b9-8231-4e10-8a70-5dee789fc9f7-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:06:25.426334 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.426293    5586 generic.go:356] "Generic (PLEG): container finished" podID="08f613b9-8231-4e10-8a70-5dee789fc9f7" containerID="15c3e146d1d909f20a7aa68356811347214756397e7946f7cc4a656850c1f6b6" exitCode=0
Oct 30 22:06:25.426513 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.426349    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"
Oct 30 22:06:25.426513 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.426374    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp" event={"ID":"08f613b9-8231-4e10-8a70-5dee789fc9f7","Type":"ContainerDied","Data":"15c3e146d1d909f20a7aa68356811347214756397e7946f7cc4a656850c1f6b6"}
Oct 30 22:06:25.426513 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.426429    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp" event={"ID":"08f613b9-8231-4e10-8a70-5dee789fc9f7","Type":"ContainerDied","Data":"7be2ce43612c0b1d93e1ddb9af197e93a3086138bd85080870e25338b372a98f"}
Oct 30 22:06:25.426513 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.426452    5586 scope.go:122] "RemoveContainer" containerID="15c3e146d1d909f20a7aa68356811347214756397e7946f7cc4a656850c1f6b6"
Oct 30 22:06:25.439167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.439146    5586 scope.go:122] "RemoveContainer" containerID="c1f958f7ae0aa57c8a5553f56d2d8e13cffbca941725d8bfedd50fc9a083fdb1"
Oct 30 22:06:25.450683 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.450646    5586 scope.go:122] "RemoveContainer" containerID="15c3e146d1d909f20a7aa68356811347214756397e7946f7cc4a656850c1f6b6"
Oct 30 22:06:25.450987 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:06:25.450965    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15c3e146d1d909f20a7aa68356811347214756397e7946f7cc4a656850c1f6b6\": container with ID starting with 15c3e146d1d909f20a7aa68356811347214756397e7946f7cc4a656850c1f6b6 not found: ID does not exist" containerID="15c3e146d1d909f20a7aa68356811347214756397e7946f7cc4a656850c1f6b6"
Oct 30 22:06:25.451072 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.450995    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15c3e146d1d909f20a7aa68356811347214756397e7946f7cc4a656850c1f6b6"} err="failed to get container status \"15c3e146d1d909f20a7aa68356811347214756397e7946f7cc4a656850c1f6b6\": rpc error: code = NotFound desc = could not find container \"15c3e146d1d909f20a7aa68356811347214756397e7946f7cc4a656850c1f6b6\": container with ID starting with 15c3e146d1d909f20a7aa68356811347214756397e7946f7cc4a656850c1f6b6 not found: ID does not exist"
Oct 30 22:06:25.451072 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.451016    5586 scope.go:122] "RemoveContainer" containerID="c1f958f7ae0aa57c8a5553f56d2d8e13cffbca941725d8bfedd50fc9a083fdb1"
Oct 30 22:06:25.451325 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:06:25.451305    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1f958f7ae0aa57c8a5553f56d2d8e13cffbca941725d8bfedd50fc9a083fdb1\": container with ID starting with c1f958f7ae0aa57c8a5553f56d2d8e13cffbca941725d8bfedd50fc9a083fdb1 not found: ID does not exist" containerID="c1f958f7ae0aa57c8a5553f56d2d8e13cffbca941725d8bfedd50fc9a083fdb1"
Oct 30 22:06:25.451378 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.451336    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1f958f7ae0aa57c8a5553f56d2d8e13cffbca941725d8bfedd50fc9a083fdb1"} err="failed to get container status \"c1f958f7ae0aa57c8a5553f56d2d8e13cffbca941725d8bfedd50fc9a083fdb1\": rpc error: code = NotFound desc = could not find container \"c1f958f7ae0aa57c8a5553f56d2d8e13cffbca941725d8bfedd50fc9a083fdb1\": container with ID starting with c1f958f7ae0aa57c8a5553f56d2d8e13cffbca941725d8bfedd50fc9a083fdb1 not found: ID does not exist"
Oct 30 22:06:25.593146 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.593055    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"]
Oct 30 22:06:25.597010 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:25.596982    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e2deec91-openshift-default-67df997f57-rlkqp"]
Oct 30 22:06:26.561929 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:26.561890    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:27.338169 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:27.338135    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:27.449355 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:27.449322    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="08f613b9-8231-4e10-8a70-5dee789fc9f7" path="/var/lib/kubelet/pods/08f613b9-8231-4e10-8a70-5dee789fc9f7/volumes"
Oct 30 22:06:32.059524 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:32.059492    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:36.980932 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:36.980897    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:37.205699 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:37.205659    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:37.376205 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:37.376085    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:37.537446 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:37.537403    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:40.408409 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:40.408364    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:40.614405 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:40.614367    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:42.465852 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:42.465814    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:47.456005 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:47.455965    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:48.366964 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:06:48.366915    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:06:48.483505 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:48.483472    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:52.502480 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:52.502437    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:52.801081 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:52.801055    5586 generic.go:356] "Generic (PLEG): container finished" podID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerID="7b783f94d3d7052180811ae2d6114920f2987a6e2c6aec2c0eba42d09fde7c2c" exitCode=0
Oct 30 22:06:52.801204 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:52.801125    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw" event={"ID":"02b0df12-ffa4-4f62-b12d-fd1e2808a730","Type":"ContainerDied","Data":"7b783f94d3d7052180811ae2d6114920f2987a6e2c6aec2c0eba42d09fde7c2c"}
Oct 30 22:06:52.801641 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:52.801623    5586 scope.go:122] "RemoveContainer" containerID="7b783f94d3d7052180811ae2d6114920f2987a6e2c6aec2c0eba42d09fde7c2c"
Oct 30 22:06:52.803125 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:52.803098    5586 generic.go:356] "Generic (PLEG): container finished" podID="6a144510-a70d-4553-8717-b048d64094b3" containerID="b842587bdde38aae6135e1d4d5757ec25c2c859789ed08dbf1b215d1390f49d3" exitCode=0
Oct 30 22:06:52.803221 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:52.803162    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz" event={"ID":"6a144510-a70d-4553-8717-b048d64094b3","Type":"ContainerDied","Data":"b842587bdde38aae6135e1d4d5757ec25c2c859789ed08dbf1b215d1390f49d3"}
Oct 30 22:06:52.803543 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:52.803517    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:06:52.803613 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:52.803576    5586 scope.go:122] "RemoveContainer" containerID="b842587bdde38aae6135e1d4d5757ec25c2c859789ed08dbf1b215d1390f49d3"
Oct 30 22:06:52.803613 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:52.803582    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:06:52.803689 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:52.803617    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:06:52.805160 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:52.805132    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:06:52.805265 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:52.805195    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:06:52.805265 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:52.805226    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:06:53.600099 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:53.598778    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:53.813343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:53.813295    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz" event={"ID":"6a144510-a70d-4553-8717-b048d64094b3","Type":"ContainerStarted","Data":"b3eef186160acd26444b392b949e5fa63e0d3d47a15c01b80d9abae3181fa070"}
Oct 30 22:06:53.815751 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:53.815730    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw" event={"ID":"02b0df12-ffa4-4f62-b12d-fd1e2808a730","Type":"ContainerStarted","Data":"8ef13f21394fa37fe8519023b6e70613b2ee6beb1dd730df47293133d7a0db10"}
Oct 30 22:06:53.969273 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:53.966922    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:06:53.976268 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:53.973356    5586 patch_prober.go:29] interesting pod/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 22:06:53.976268 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:53.973416    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw" podUID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 22:06:54.401306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.401204    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:06:54.401498 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.401433    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 30 22:06:54.401569 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.401486    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz" podUID="6a144510-a70d-4553-8717-b048d64094b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 30 22:06:54.800039 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.799993    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"]
Oct 30 22:06:54.811227 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.811195    5586 cpu_manager.go:414] "containerMap: removing container" podUID="08f613b9-8231-4e10-8a70-5dee789fc9f7" containerName="istio-proxy"
Oct 30 22:06:54.811477 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.811457    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08f613b9-8231-4e10-8a70-5dee789fc9f7" containerName="istio-proxy"
Oct 30 22:06:54.811627 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.811614    5586 cpu_manager.go:414] "containerMap: removing container" podUID="08f613b9-8231-4e10-8a70-5dee789fc9f7" containerName="istio-proxy"
Oct 30 22:06:54.811717 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.811707    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08f613b9-8231-4e10-8a70-5dee789fc9f7" containerName="istio-proxy"
Oct 30 22:06:54.812190 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.812169    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="08f613b9-8231-4e10-8a70-5dee789fc9f7" containerName="istio-proxy"
Oct 30 22:06:54.812467 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.812454    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="08f613b9-8231-4e10-8a70-5dee789fc9f7" containerName="istio-proxy"
Oct 30 22:06:54.817268 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.817216    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:54.917265 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.905949    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-78cf83-openshift-default-dockercfg-8qjhq\""
Oct 30 22:06:54.946406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.933738    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:54.989038 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.988886    5586 patch_prober.go:29] interesting pod/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 22:06:54.989375 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:54.989339    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw" podUID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 22:06:55.006281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.000316    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"]
Oct 30 22:06:55.078119 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.078010    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aae92a23-65e7-4f07-bbf8-934cc18e916e-istiod-ca-cert\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.078324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.078194    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-data\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.078324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.078278    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-token\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.078324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.078317    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-workload-socket\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.078499 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.078490    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-workload-certs\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.078566 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.078540    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-envoy\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.078627 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.078608    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-podinfo\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.078688 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.078648    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-credential-socket\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.078757 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.078692    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq8t6\" (UniqueName: \"kubernetes.io/projected/aae92a23-65e7-4f07-bbf8-934cc18e916e-kube-api-access-bq8t6\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.180420 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.180373    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-data\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.180639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.180464    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-token\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.180639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.180522    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-workload-socket\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.180639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.180559    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-workload-certs\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.180639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.180604    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-envoy\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.180887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.180705    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-podinfo\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.180887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.180778    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-credential-socket\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.180887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.180821    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq8t6\" (UniqueName: \"kubernetes.io/projected/aae92a23-65e7-4f07-bbf8-934cc18e916e-kube-api-access-bq8t6\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.181062 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.180908    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aae92a23-65e7-4f07-bbf8-934cc18e916e-istiod-ca-cert\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.181309 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.181270    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-workload-certs\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.181501 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.181475    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-data\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.181702 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.181677    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-workload-socket\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.181837 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.181814    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-credential-socket\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.182497 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.182470    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aae92a23-65e7-4f07-bbf8-934cc18e916e-istiod-ca-cert\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.184073 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.184047    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-envoy\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.184861 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.184837    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-podinfo\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.263067 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.263031    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-token\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.287276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.287151    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq8t6\" (UniqueName: \"kubernetes.io/projected/aae92a23-65e7-4f07-bbf8-934cc18e916e-kube-api-access-bq8t6\") pod \"e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") " pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.292011 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.291973    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:55.401963 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.401860    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 30 22:06:55.401963 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.401951    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz" podUID="6a144510-a70d-4553-8717-b048d64094b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 30 22:06:55.453257 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.452526    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:55.970264 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.965802    5586 patch_prober.go:29] interesting pod/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 22:06:55.970264 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.965869    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw" podUID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 22:06:55.980787 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.980754    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"]
Oct 30 22:06:55.986528 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.986469    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:06:55.986671 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.986564    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:06:55.986671 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:55.986610    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:06:56.402329 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:56.401829    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 30 22:06:56.402329 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:56.401885    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz" podUID="6a144510-a70d-4553-8717-b048d64094b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 30 22:06:56.971292 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:56.969741    5586 patch_prober.go:29] interesting pod/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 22:06:56.971292 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:56.969820    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw" podUID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 22:06:56.988479 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:56.988440    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb" event={"ID":"aae92a23-65e7-4f07-bbf8-934cc18e916e","Type":"ContainerStarted","Data":"e0b7d75fd324d899b6843219845204ebdf3e2331f7623f90d1078d603337631d"}
Oct 30 22:06:56.988684 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:56.988503    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb" event={"ID":"aae92a23-65e7-4f07-bbf8-934cc18e916e","Type":"ContainerStarted","Data":"a0c08a8859824feaa01ffb9d6885c8692e1e2620b4385933e0551daf0398e969"}
Oct 30 22:06:57.110643 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:57.110578    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb" podStartSLOduration=3.110562288 podStartE2EDuration="3.110562288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:06:57.109031214 +0000 UTC m=+1959.780935149" watchObservedRunningTime="2026-10-30 22:06:57.110562288 +0000 UTC m=+1959.782466225"
Oct 30 22:06:57.118499 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:57.118467    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:06:57.377816 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:57.377770    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:06:57.402827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:57.401853    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 30 22:06:57.402827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:57.401918    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz" podUID="6a144510-a70d-4553-8717-b048d64094b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 30 22:06:57.495968 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:57.495936    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:57.496125 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:57.496034    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:57.968255 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:57.967698    5586 patch_prober.go:29] interesting pod/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 22:06:57.968255 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:57.967765    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw" podUID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 22:06:58.000272 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:57.997655    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:58.000272 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:57.998713    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:06:58.401808 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:58.401716    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 30 22:06:58.401808 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:58.401779    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz" podUID="6a144510-a70d-4553-8717-b048d64094b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 30 22:06:58.974261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:58.973774    5586 patch_prober.go:29] interesting pod/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 22:06:58.974261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:58.973884    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw" podUID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 22:06:59.489775 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:59.489740    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:06:59.490501 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:59.490464    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:06:59.491464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:59.491441    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:06:59.970868 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:59.970838    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:06:59.971888 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:59.971865    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:06:59.984777 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:06:59.984748    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:00.230824 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:00.230634    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:02.482194 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:02.482159    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:03.195804 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:03.195762    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:03.369138 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:03.369097    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:07.709661 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:07.709622    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:12.483899 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:12.483853    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:17.221097 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:17.221064    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:17.372762 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:17.372725    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:17.734609 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:17.734570    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:18.171717 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:18.171681    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:18.362676 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:07:18.362631    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:07:18.917395 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:18.917350    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:19.587193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:19.587156    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:19.992531 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:19.992494    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:23.024337 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:23.024294    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:24.345193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:24.345154    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:24.531566 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:24.531525    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:24.695725 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:24.695682    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:26.906770 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:26.906727    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:27.860990 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:27.860950    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"]
Oct 30 22:07:27.866314 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:27.866281    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:27.922078 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:27.922039    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-6cd200-serving-cert\""
Oct 30 22:07:27.978718 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:27.978673    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e584555-1a5c-429b-81b2-be625654d315-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:27.980796 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:27.978731    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e584555-1a5c-429b-81b2-be625654d315-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:27.980796 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:27.979027    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjrkq\" (UniqueName: \"kubernetes.io/projected/1e584555-1a5c-429b-81b2-be625654d315-kube-api-access-wjrkq\") pod \"maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:27.980796 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:27.979212    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e584555-1a5c-429b-81b2-be625654d315-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:27.985433 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:27.985398    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"]
Oct 30 22:07:28.080313 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:28.080271    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e584555-1a5c-429b-81b2-be625654d315-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:28.080313 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:28.080314    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e584555-1a5c-429b-81b2-be625654d315-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:28.080583 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:28.080387    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjrkq\" (UniqueName: \"kubernetes.io/projected/1e584555-1a5c-429b-81b2-be625654d315-kube-api-access-wjrkq\") pod \"maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:28.080583 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:28.080496    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e584555-1a5c-429b-81b2-be625654d315-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:28.081163 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:28.081140    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e584555-1a5c-429b-81b2-be625654d315-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:28.083119 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:28.083091    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e584555-1a5c-429b-81b2-be625654d315-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:28.083273 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:28.083160    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e584555-1a5c-429b-81b2-be625654d315-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:28.138278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:28.138166    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjrkq\" (UniqueName: \"kubernetes.io/projected/1e584555-1a5c-429b-81b2-be625654d315-kube-api-access-wjrkq\") pod \"maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:28.184404 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:28.184370    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:28.587954 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:28.587851    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:28.591276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:28.591244    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"]
Oct 30 22:07:28.593353 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:07:28.593325    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e584555_1a5c_429b_81b2_be625654d315.slice/crio-eb948972db4424753d05195190e5516fc68c8c8f826c0cd6e62edc86baf91064 WatchSource:0}: Error finding container eb948972db4424753d05195190e5516fc68c8c8f826c0cd6e62edc86baf91064: Status 404 returned error can't find the container with id eb948972db4424753d05195190e5516fc68c8c8f826c0cd6e62edc86baf91064
Oct 30 22:07:29.419381 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:29.419332    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd" event={"ID":"1e584555-1a5c-429b-81b2-be625654d315","Type":"ContainerStarted","Data":"eb948972db4424753d05195190e5516fc68c8c8f826c0cd6e62edc86baf91064"}
Oct 30 22:07:30.430371 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:30.430329    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd" event={"ID":"1e584555-1a5c-429b-81b2-be625654d315","Type":"ContainerStarted","Data":"ce486372b28b4458dbfc4f924c4c009ff6a88e4d77dc32dc7b86060462f9c7fd"}
Oct 30 22:07:30.430830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:30.430557    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:30.551146 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:30.551081    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd" podStartSLOduration=2.664480305 podStartE2EDuration="3.551063256s" totalImagesPullingTime="886.582951ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:30.545924268 +0000 UTC m=+1993.217828204" watchObservedRunningTime="2026-10-30 22:07:30.551063256 +0000 UTC m=+1993.222967190"
Oct 30 22:07:33.573388 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:33.573342    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:36.444955 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:36.444922    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:07:38.299334 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:38.299288    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:43.089722 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:43.089684    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:48.318056 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:48.318014    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:48.398734 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:07:48.398694    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:07:53.598431 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:53.598393    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:58.765673 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:58.765639    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:07:59.713930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:59.713838    5586 generic.go:356] "Generic (PLEG): container finished" podID="89b841c7-c736-4ffe-b31e-762908d62361" containerID="a410127d6bfb122a0f4875cd14b5d6737ebce0c77e381e0d91f7e61b551bca62" exitCode=0
Oct 30 22:07:59.714162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:59.713919    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h" event={"ID":"89b841c7-c736-4ffe-b31e-762908d62361","Type":"ContainerDied","Data":"a410127d6bfb122a0f4875cd14b5d6737ebce0c77e381e0d91f7e61b551bca62"}
Oct 30 22:07:59.714450 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:59.714431    5586 scope.go:122] "RemoveContainer" containerID="a410127d6bfb122a0f4875cd14b5d6737ebce0c77e381e0d91f7e61b551bca62"
Oct 30 22:07:59.716368 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:59.716342    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:07:59.716444 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:59.716412    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:07:59.716481 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:07:59.716444    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:08:00.218476 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:00.218440    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:00.747880 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:00.747841    5586 generic.go:356] "Generic (PLEG): container finished" podID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerID="183b7260b4d2ee4c0c91cde2927605d1dbe4d7065288829b79369a7bc3ad52f6" exitCode=0
Oct 30 22:08:00.748072 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:00.747917    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx" event={"ID":"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7","Type":"ContainerDied","Data":"183b7260b4d2ee4c0c91cde2927605d1dbe4d7065288829b79369a7bc3ad52f6"}
Oct 30 22:08:00.748532 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:00.748512    5586 scope.go:122] "RemoveContainer" containerID="183b7260b4d2ee4c0c91cde2927605d1dbe4d7065288829b79369a7bc3ad52f6"
Oct 30 22:08:00.750485 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:00.750460    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h" event={"ID":"89b841c7-c736-4ffe-b31e-762908d62361","Type":"ContainerStarted","Data":"4147b489f9b23333acbd87d6e29fca226e7436395d5cfb1881daef617b635142"}
Oct 30 22:08:00.750591 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:00.750503    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:08:00.750670 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:00.750590    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:08:00.750670 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:00.750642    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:08:01.548160 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:01.548120    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:08:01.548679 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:01.548646    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:08:01.548784 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:01.548713    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h" podUID="89b841c7-c736-4ffe-b31e-762908d62361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 22:08:01.782345 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:01.782293    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx" event={"ID":"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7","Type":"ContainerStarted","Data":"1d7a06da2d3873c2d13d528d0787be2f99d9074842b56170b972c53a370d9158"}
Oct 30 22:08:01.783861 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:01.783832    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:08:01.784997 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:01.784972    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 22:08:01.785113 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:01.785022    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx" podUID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 22:08:02.567259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:02.566875    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:08:02.567259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:02.566943    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h" podUID="89b841c7-c736-4ffe-b31e-762908d62361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 22:08:02.573486 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:02.568651    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"]
Oct 30 22:08:02.683671 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:02.683611    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"]
Oct 30 22:08:02.784528 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:02.784486    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 22:08:02.784695 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:02.784566    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx" podUID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 22:08:02.958620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:02.958483    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:03.157260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:03.157185    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:03.549260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:03.548724    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:08:03.549260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:03.548807    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h" podUID="89b841c7-c736-4ffe-b31e-762908d62361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 22:08:03.784902 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:03.784849    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 22:08:03.785458 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:03.784918    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx" podUID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 22:08:03.810745 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:03.810657    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:03.885258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:03.884346    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:08:04.547257 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:04.546640    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:08:04.547257 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:04.546715    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h" podUID="89b841c7-c736-4ffe-b31e-762908d62361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 22:08:04.784203 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:04.784160    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 22:08:04.784427 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:04.784219    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx" podUID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 22:08:04.887919 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:04.887833    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:08:05.546795 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:05.546757    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:08:05.547117 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:05.547085    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h" podUID="89b841c7-c736-4ffe-b31e-762908d62361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 22:08:05.783959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:05.783915    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 22:08:05.784176 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:05.783976    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx" podUID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 22:08:06.596192 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:06.596148    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:08:06.597880 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:06.597857    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:08:06.783802 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:06.783766    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 22:08:06.784003 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:06.783823    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx" podUID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 22:08:07.569757 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.569702    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn" podUID="dd404a46-bc6b-4a63-bd7c-e09793a2e377" containerName="payload-processing" containerID="cri-o://061a4484e6404e2697e164be52f5e6f2a54ef19e902663ef2923c9b592732d42" gracePeriod=25
Oct 30 22:08:07.684792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.684740    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4" podUID="b9c7ab71-5d38-4489-95fe-960b27c7bf40" containerName="payload-pre-processing" containerID="cri-o://b1663a041010bb706780dfdaa3d1445b8f1f528b7f0f791140a61ae3f99b7e35" gracePeriod=25
Oct 30 22:08:07.792104 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.791837    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:08:07.794705 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.794679    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:08:07.840143 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.840067    5586 generic.go:356] "Generic (PLEG): container finished" podID="dd404a46-bc6b-4a63-bd7c-e09793a2e377" containerID="061a4484e6404e2697e164be52f5e6f2a54ef19e902663ef2923c9b592732d42" exitCode=0
Oct 30 22:08:07.840813 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.840420    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn" event={"ID":"dd404a46-bc6b-4a63-bd7c-e09793a2e377","Type":"ContainerDied","Data":"061a4484e6404e2697e164be52f5e6f2a54ef19e902663ef2923c9b592732d42"}
Oct 30 22:08:07.843329 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.842424    5586 generic.go:356] "Generic (PLEG): container finished" podID="b9c7ab71-5d38-4489-95fe-960b27c7bf40" containerID="b1663a041010bb706780dfdaa3d1445b8f1f528b7f0f791140a61ae3f99b7e35" exitCode=0
Oct 30 22:08:07.843329 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.843276    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4" event={"ID":"b9c7ab71-5d38-4489-95fe-960b27c7bf40","Type":"ContainerDied","Data":"b1663a041010bb706780dfdaa3d1445b8f1f528b7f0f791140a61ae3f99b7e35"}
Oct 30 22:08:07.888582 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.888547    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"
Oct 30 22:08:07.920680 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.920646    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd404a46-bc6b-4a63-bd7c-e09793a2e377-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd404a46-bc6b-4a63-bd7c-e09793a2e377-plugins-config-volume\") pod \"dd404a46-bc6b-4a63-bd7c-e09793a2e377\" (UID: \"dd404a46-bc6b-4a63-bd7c-e09793a2e377\") "
Oct 30 22:08:07.920882 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.920703    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd404a46-bc6b-4a63-bd7c-e09793a2e377-kube-api-access-5fgxr\" (UniqueName: \"kubernetes.io/projected/dd404a46-bc6b-4a63-bd7c-e09793a2e377-kube-api-access-5fgxr\") pod \"dd404a46-bc6b-4a63-bd7c-e09793a2e377\" (UID: \"dd404a46-bc6b-4a63-bd7c-e09793a2e377\") "
Oct 30 22:08:07.921260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.921216    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd404a46-bc6b-4a63-bd7c-e09793a2e377-plugins-config-volume" pod "dd404a46-bc6b-4a63-bd7c-e09793a2e377" (UID: "dd404a46-bc6b-4a63-bd7c-e09793a2e377"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:07.921628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.921600    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd404a46-bc6b-4a63-bd7c-e09793a2e377-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:07.923627 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.923602    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd404a46-bc6b-4a63-bd7c-e09793a2e377-kube-api-access-5fgxr" pod "dd404a46-bc6b-4a63-bd7c-e09793a2e377" (UID: "dd404a46-bc6b-4a63-bd7c-e09793a2e377"). InnerVolumeSpecName "kube-api-access-5fgxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:07.968719 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:07.968693    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"
Oct 30 22:08:08.022171 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:08.022139    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9c7ab71-5d38-4489-95fe-960b27c7bf40-kube-api-access-72v86\" (UniqueName: \"kubernetes.io/projected/b9c7ab71-5d38-4489-95fe-960b27c7bf40-kube-api-access-72v86\") pod \"b9c7ab71-5d38-4489-95fe-960b27c7bf40\" (UID: \"b9c7ab71-5d38-4489-95fe-960b27c7bf40\") "
Oct 30 22:08:08.022409 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:08.022340    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9c7ab71-5d38-4489-95fe-960b27c7bf40-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9c7ab71-5d38-4489-95fe-960b27c7bf40-plugins-config-volume\") pod \"b9c7ab71-5d38-4489-95fe-960b27c7bf40\" (UID: \"b9c7ab71-5d38-4489-95fe-960b27c7bf40\") "
Oct 30 22:08:08.022715 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:08.022689    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9c7ab71-5d38-4489-95fe-960b27c7bf40-plugins-config-volume" pod "b9c7ab71-5d38-4489-95fe-960b27c7bf40" (UID: "b9c7ab71-5d38-4489-95fe-960b27c7bf40"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:08.022873 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:08.022853    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9c7ab71-5d38-4489-95fe-960b27c7bf40-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:08.022929 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:08.022879    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fgxr\" (UniqueName: \"kubernetes.io/projected/dd404a46-bc6b-4a63-bd7c-e09793a2e377-kube-api-access-5fgxr\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:08.024270 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:08.024250    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9c7ab71-5d38-4489-95fe-960b27c7bf40-kube-api-access-72v86" pod "b9c7ab71-5d38-4489-95fe-960b27c7bf40" (UID: "b9c7ab71-5d38-4489-95fe-960b27c7bf40"). InnerVolumeSpecName "kube-api-access-72v86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:08.124160 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:08.124082    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72v86\" (UniqueName: \"kubernetes.io/projected/b9c7ab71-5d38-4489-95fe-960b27c7bf40-kube-api-access-72v86\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:08.851264 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:08.851165    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"
Oct 30 22:08:08.851264 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:08.851198    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4" event={"ID":"b9c7ab71-5d38-4489-95fe-960b27c7bf40","Type":"ContainerDied","Data":"b59c574de20210a2e5b7de663044fbcb436e95a1a6ff92c86d953bad4818f940"}
Oct 30 22:08:08.851836 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:08.851272    5586 scope.go:122] "RemoveContainer" containerID="b1663a041010bb706780dfdaa3d1445b8f1f528b7f0f791140a61ae3f99b7e35"
Oct 30 22:08:08.853015 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:08.852975    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn" event={"ID":"dd404a46-bc6b-4a63-bd7c-e09793a2e377","Type":"ContainerDied","Data":"b49095181d4160ca23a9f754e7264a47f9f0711beb5585c976b9db2a577937d2"}
Oct 30 22:08:08.853116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:08.853080    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"
Oct 30 22:08:08.865089 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:08.865065    5586 scope.go:122] "RemoveContainer" containerID="061a4484e6404e2697e164be52f5e6f2a54ef19e902663ef2923c9b592732d42"
Oct 30 22:08:09.030860 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:09.030827    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"]
Oct 30 22:08:09.037355 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:09.037324    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-99f69c-76b8547dfc-q87qn"]
Oct 30 22:08:09.147144 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:09.147064    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"]
Oct 30 22:08:09.159479 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:09.159430    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-99f69c-bb7689b7f-jjlw4"]
Oct 30 22:08:09.250129 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:09.250078    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:09.416807 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:09.416768    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:09.468975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:09.468936    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9c7ab71-5d38-4489-95fe-960b27c7bf40" path="/var/lib/kubelet/pods/b9c7ab71-5d38-4489-95fe-960b27c7bf40/volumes"
Oct 30 22:08:09.469438 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:09.469423    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd404a46-bc6b-4a63-bd7c-e09793a2e377" path="/var/lib/kubelet/pods/dd404a46-bc6b-4a63-bd7c-e09793a2e377/volumes"
Oct 30 22:08:09.574970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:09.574934    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:14.260829 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:14.260798    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:14.551093 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:14.550919    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"]
Oct 30 22:08:14.662857 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:14.662819    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"]
Oct 30 22:08:14.931601 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:14.931556    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:15.083849 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:15.083438    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:18.364141 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:08:18.364101    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:08:19.081722 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:19.081692    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:19.551745 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:19.551699    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n" podUID="7b8ea9ca-9405-4abf-852c-04a8289fb6ec" containerName="payload-processing" containerID="cri-o://9b1c0634616f5de3eac94e79297aee28b25d177d5a528a65521c7349f98ac2d5" gracePeriod=25
Oct 30 22:08:19.663698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:19.663655    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll" podUID="4ec4ead0-ee39-4c04-8512-716f6143b147" containerName="payload-pre-processing" containerID="cri-o://cb751c94cf26aa6b65e3b19f7baf4e79d9e698a6e0447f20c227dd1f92dd45bb" gracePeriod=25
Oct 30 22:08:23.846744 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:23.846689    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:26.783074 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:26.783038    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll" podUID="4ec4ead0-ee39-4c04-8512-716f6143b147" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:08:26.783525 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:26.783046    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n" podUID="7b8ea9ca-9405-4abf-852c-04a8289fb6ec" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:08:27.241069 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:27.241036    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:27.402172 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:27.402137    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:28.586497 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:28.586456    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:33.415625 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:33.415581    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:36.783450 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:36.783407    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll" podUID="4ec4ead0-ee39-4c04-8512-716f6143b147" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:08:36.783855 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:36.783407    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n" podUID="7b8ea9ca-9405-4abf-852c-04a8289fb6ec" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:08:36.827384 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:36.827347    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"]
Oct 30 22:08:36.827597 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:36.827576    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" podUID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" containerName="istio-proxy" containerID="cri-o://cb8d9ac20498f562a95b04a572c27bff38ee85cd9b5abb150b9f215a613449d7" gracePeriod=30
Oct 30 22:08:37.295651 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:37.295615    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:37.446636 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:37.446598    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:38.123479 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:38.123439    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:38.289042 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:38.288997    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:39.836122 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:39.836088    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"
Oct 30 22:08:39.942569 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:39.942542    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b8ea9ca-9405-4abf-852c-04a8289fb6ec-kube-api-access-74x8z\" (UniqueName: \"kubernetes.io/projected/7b8ea9ca-9405-4abf-852c-04a8289fb6ec-kube-api-access-74x8z\") pod \"7b8ea9ca-9405-4abf-852c-04a8289fb6ec\" (UID: \"7b8ea9ca-9405-4abf-852c-04a8289fb6ec\") "
Oct 30 22:08:39.942732 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:39.942704    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7b8ea9ca-9405-4abf-852c-04a8289fb6ec-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b8ea9ca-9405-4abf-852c-04a8289fb6ec-plugins-config-volume\") pod \"7b8ea9ca-9405-4abf-852c-04a8289fb6ec\" (UID: \"7b8ea9ca-9405-4abf-852c-04a8289fb6ec\") "
Oct 30 22:08:39.943016 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:39.942991    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b8ea9ca-9405-4abf-852c-04a8289fb6ec-plugins-config-volume" pod "7b8ea9ca-9405-4abf-852c-04a8289fb6ec" (UID: "7b8ea9ca-9405-4abf-852c-04a8289fb6ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:39.943320 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:39.943301    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b8ea9ca-9405-4abf-852c-04a8289fb6ec-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:39.944809 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:39.944787    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b8ea9ca-9405-4abf-852c-04a8289fb6ec-kube-api-access-74x8z" pod "7b8ea9ca-9405-4abf-852c-04a8289fb6ec" (UID: "7b8ea9ca-9405-4abf-852c-04a8289fb6ec"). InnerVolumeSpecName "kube-api-access-74x8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:39.954424 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:39.954396    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"
Oct 30 22:08:40.044031 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.043995    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ec4ead0-ee39-4c04-8512-716f6143b147-kube-api-access-jw2v9\" (UniqueName: \"kubernetes.io/projected/4ec4ead0-ee39-4c04-8512-716f6143b147-kube-api-access-jw2v9\") pod \"4ec4ead0-ee39-4c04-8512-716f6143b147\" (UID: \"4ec4ead0-ee39-4c04-8512-716f6143b147\") "
Oct 30 22:08:40.044227 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.044214    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4ec4ead0-ee39-4c04-8512-716f6143b147-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ec4ead0-ee39-4c04-8512-716f6143b147-plugins-config-volume\") pod \"4ec4ead0-ee39-4c04-8512-716f6143b147\" (UID: \"4ec4ead0-ee39-4c04-8512-716f6143b147\") "
Oct 30 22:08:40.044610 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.044586    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74x8z\" (UniqueName: \"kubernetes.io/projected/7b8ea9ca-9405-4abf-852c-04a8289fb6ec-kube-api-access-74x8z\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:40.044610 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.044596    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ec4ead0-ee39-4c04-8512-716f6143b147-plugins-config-volume" pod "4ec4ead0-ee39-4c04-8512-716f6143b147" (UID: "4ec4ead0-ee39-4c04-8512-716f6143b147"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:40.046130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.046107    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ec4ead0-ee39-4c04-8512-716f6143b147-kube-api-access-jw2v9" pod "4ec4ead0-ee39-4c04-8512-716f6143b147" (UID: "4ec4ead0-ee39-4c04-8512-716f6143b147"). InnerVolumeSpecName "kube-api-access-jw2v9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:40.130722 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.130625    5586 generic.go:356] "Generic (PLEG): container finished" podID="4ec4ead0-ee39-4c04-8512-716f6143b147" containerID="cb751c94cf26aa6b65e3b19f7baf4e79d9e698a6e0447f20c227dd1f92dd45bb" exitCode=0
Oct 30 22:08:40.130722 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.130681    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"
Oct 30 22:08:40.130952 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.130710    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll" event={"ID":"4ec4ead0-ee39-4c04-8512-716f6143b147","Type":"ContainerDied","Data":"cb751c94cf26aa6b65e3b19f7baf4e79d9e698a6e0447f20c227dd1f92dd45bb"}
Oct 30 22:08:40.130952 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.130780    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll" event={"ID":"4ec4ead0-ee39-4c04-8512-716f6143b147","Type":"ContainerDied","Data":"32f0502a3fac68ef3177d7f13db703331f0602bbad06ef498128715165ea6bea"}
Oct 30 22:08:40.130952 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.130813    5586 scope.go:122] "RemoveContainer" containerID="cb751c94cf26aa6b65e3b19f7baf4e79d9e698a6e0447f20c227dd1f92dd45bb"
Oct 30 22:08:40.132550 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.132521    5586 generic.go:356] "Generic (PLEG): container finished" podID="7b8ea9ca-9405-4abf-852c-04a8289fb6ec" containerID="9b1c0634616f5de3eac94e79297aee28b25d177d5a528a65521c7349f98ac2d5" exitCode=0
Oct 30 22:08:40.132696 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.132547    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n" event={"ID":"7b8ea9ca-9405-4abf-852c-04a8289fb6ec","Type":"ContainerDied","Data":"9b1c0634616f5de3eac94e79297aee28b25d177d5a528a65521c7349f98ac2d5"}
Oct 30 22:08:40.132696 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.132588    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"
Oct 30 22:08:40.132696 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.132600    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n" event={"ID":"7b8ea9ca-9405-4abf-852c-04a8289fb6ec","Type":"ContainerDied","Data":"e599fa39a655adc7ce92d920f0617f022bd6e1e946d7209e056c201a151a2a04"}
Oct 30 22:08:40.143484 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.143462    5586 scope.go:122] "RemoveContainer" containerID="cb751c94cf26aa6b65e3b19f7baf4e79d9e698a6e0447f20c227dd1f92dd45bb"
Oct 30 22:08:40.143826 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:08:40.143789    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb751c94cf26aa6b65e3b19f7baf4e79d9e698a6e0447f20c227dd1f92dd45bb\": container with ID starting with cb751c94cf26aa6b65e3b19f7baf4e79d9e698a6e0447f20c227dd1f92dd45bb not found: ID does not exist" containerID="cb751c94cf26aa6b65e3b19f7baf4e79d9e698a6e0447f20c227dd1f92dd45bb"
Oct 30 22:08:40.143910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.143819    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb751c94cf26aa6b65e3b19f7baf4e79d9e698a6e0447f20c227dd1f92dd45bb"} err="failed to get container status \"cb751c94cf26aa6b65e3b19f7baf4e79d9e698a6e0447f20c227dd1f92dd45bb\": rpc error: code = NotFound desc = could not find container \"cb751c94cf26aa6b65e3b19f7baf4e79d9e698a6e0447f20c227dd1f92dd45bb\": container with ID starting with cb751c94cf26aa6b65e3b19f7baf4e79d9e698a6e0447f20c227dd1f92dd45bb not found: ID does not exist"
Oct 30 22:08:40.143910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.143843    5586 scope.go:122] "RemoveContainer" containerID="9b1c0634616f5de3eac94e79297aee28b25d177d5a528a65521c7349f98ac2d5"
Oct 30 22:08:40.145556 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.145533    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw2v9\" (UniqueName: \"kubernetes.io/projected/4ec4ead0-ee39-4c04-8512-716f6143b147-kube-api-access-jw2v9\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:40.145556 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.145556    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ec4ead0-ee39-4c04-8512-716f6143b147-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:40.154212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.154192    5586 scope.go:122] "RemoveContainer" containerID="9b1c0634616f5de3eac94e79297aee28b25d177d5a528a65521c7349f98ac2d5"
Oct 30 22:08:40.154582 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:08:40.154564    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b1c0634616f5de3eac94e79297aee28b25d177d5a528a65521c7349f98ac2d5\": container with ID starting with 9b1c0634616f5de3eac94e79297aee28b25d177d5a528a65521c7349f98ac2d5 not found: ID does not exist" containerID="9b1c0634616f5de3eac94e79297aee28b25d177d5a528a65521c7349f98ac2d5"
Oct 30 22:08:40.154650 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.154589    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b1c0634616f5de3eac94e79297aee28b25d177d5a528a65521c7349f98ac2d5"} err="failed to get container status \"9b1c0634616f5de3eac94e79297aee28b25d177d5a528a65521c7349f98ac2d5\": rpc error: code = NotFound desc = could not find container \"9b1c0634616f5de3eac94e79297aee28b25d177d5a528a65521c7349f98ac2d5\": container with ID starting with 9b1c0634616f5de3eac94e79297aee28b25d177d5a528a65521c7349f98ac2d5 not found: ID does not exist"
Oct 30 22:08:40.296311 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.296270    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"]
Oct 30 22:08:40.300217 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.300189    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6df3ba0e-7455bf6cc9-9xpll"]
Oct 30 22:08:40.407676 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.407641    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"]
Oct 30 22:08:40.411406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:40.411383    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6df3ba0e-84d7646477-f5t4n"]
Oct 30 22:08:41.448725 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:41.448689    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ec4ead0-ee39-4c04-8512-716f6143b147" path="/var/lib/kubelet/pods/4ec4ead0-ee39-4c04-8512-716f6143b147/volumes"
Oct 30 22:08:41.449161 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:41.449146    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7b8ea9ca-9405-4abf-852c-04a8289fb6ec" path="/var/lib/kubelet/pods/7b8ea9ca-9405-4abf-852c-04a8289fb6ec/volumes"
Oct 30 22:08:42.125056 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.125024    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 22:08:42.153030 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.152999    5586 generic.go:356] "Generic (PLEG): container finished" podID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" containerID="cb8d9ac20498f562a95b04a572c27bff38ee85cd9b5abb150b9f215a613449d7" exitCode=0
Oct 30 22:08:42.153220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.153043    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"
Oct 30 22:08:42.153220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.153076    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" event={"ID":"c20087fb-273e-4ba6-bc41-3a31cfeb30ec","Type":"ContainerDied","Data":"cb8d9ac20498f562a95b04a572c27bff38ee85cd9b5abb150b9f215a613449d7"}
Oct 30 22:08:42.153220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.153129    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2" event={"ID":"c20087fb-273e-4ba6-bc41-3a31cfeb30ec","Type":"ContainerDied","Data":"673fecacbed66efc276a13979026deb44b4b5ee11c8a8447c46233c036debe79"}
Oct 30 22:08:42.153220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.153154    5586 scope.go:122] "RemoveContainer" containerID="cb8d9ac20498f562a95b04a572c27bff38ee85cd9b5abb150b9f215a613449d7"
Oct 30 22:08:42.163930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.163911    5586 scope.go:122] "RemoveContainer" containerID="cb8d9ac20498f562a95b04a572c27bff38ee85cd9b5abb150b9f215a613449d7"
Oct 30 22:08:42.164210 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:08:42.164189    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb8d9ac20498f562a95b04a572c27bff38ee85cd9b5abb150b9f215a613449d7\": container with ID starting with cb8d9ac20498f562a95b04a572c27bff38ee85cd9b5abb150b9f215a613449d7 not found: ID does not exist" containerID="cb8d9ac20498f562a95b04a572c27bff38ee85cd9b5abb150b9f215a613449d7"
Oct 30 22:08:42.164307 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.164217    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb8d9ac20498f562a95b04a572c27bff38ee85cd9b5abb150b9f215a613449d7"} err="failed to get container status \"cb8d9ac20498f562a95b04a572c27bff38ee85cd9b5abb150b9f215a613449d7\": rpc error: code = NotFound desc = could not find container \"cb8d9ac20498f562a95b04a572c27bff38ee85cd9b5abb150b9f215a613449d7\": container with ID starting with cb8d9ac20498f562a95b04a572c27bff38ee85cd9b5abb150b9f215a613449d7 not found: ID does not exist"
Oct 30 22:08:42.175282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.175258    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-kube-api-access-9h2tp\" (UniqueName: \"kubernetes.io/projected/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-kube-api-access-9h2tp\") pod \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") "
Oct 30 22:08:42.175400 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.175314    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-podinfo\") pod \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") "
Oct 30 22:08:42.175400 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.175386    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-workload-certs\") pod \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") "
Oct 30 22:08:42.175479 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.175415    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-credential-socket\") pod \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") "
Oct 30 22:08:42.175547 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.175534    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-token\" (UniqueName: \"kubernetes.io/projected/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-token\") pod \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") "
Oct 30 22:08:42.175603 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.175554    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-envoy\") pod \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") "
Oct 30 22:08:42.175661 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.175612    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-workload-socket\") pod \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") "
Oct 30 22:08:42.175777 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.175699    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-workload-certs" pod "c20087fb-273e-4ba6-bc41-3a31cfeb30ec" (UID: "c20087fb-273e-4ba6-bc41-3a31cfeb30ec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:42.175777 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.175713    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-data\") pod \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") "
Oct 30 22:08:42.175910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.175778    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-credential-socket" pod "c20087fb-273e-4ba6-bc41-3a31cfeb30ec" (UID: "c20087fb-273e-4ba6-bc41-3a31cfeb30ec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:42.175910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.175856    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istiod-ca-cert\") pod \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\" (UID: \"c20087fb-273e-4ba6-bc41-3a31cfeb30ec\") "
Oct 30 22:08:42.176022 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.175957    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-data" pod "c20087fb-273e-4ba6-bc41-3a31cfeb30ec" (UID: "c20087fb-273e-4ba6-bc41-3a31cfeb30ec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:42.176022 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.175973    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-workload-socket" pod "c20087fb-273e-4ba6-bc41-3a31cfeb30ec" (UID: "c20087fb-273e-4ba6-bc41-3a31cfeb30ec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:42.176212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.176188    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istiod-ca-cert" pod "c20087fb-273e-4ba6-bc41-3a31cfeb30ec" (UID: "c20087fb-273e-4ba6-bc41-3a31cfeb30ec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:42.176688 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.176668    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:42.176764 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.176695    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:42.176764 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.176710    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:42.176764 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.176724    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:42.176764 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.176739    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:42.177743 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.177717    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-kube-api-access-9h2tp" pod "c20087fb-273e-4ba6-bc41-3a31cfeb30ec" (UID: "c20087fb-273e-4ba6-bc41-3a31cfeb30ec"). InnerVolumeSpecName "kube-api-access-9h2tp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:42.178021 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.178002    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-envoy" pod "c20087fb-273e-4ba6-bc41-3a31cfeb30ec" (UID: "c20087fb-273e-4ba6-bc41-3a31cfeb30ec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:42.178099 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.178044    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-podinfo" pod "c20087fb-273e-4ba6-bc41-3a31cfeb30ec" (UID: "c20087fb-273e-4ba6-bc41-3a31cfeb30ec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:08:42.178156 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.178101    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-token" pod "c20087fb-273e-4ba6-bc41-3a31cfeb30ec" (UID: "c20087fb-273e-4ba6-bc41-3a31cfeb30ec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:42.278142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.278099    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:42.278142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.278138    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:42.278394 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.278153    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h2tp\" (UniqueName: \"kubernetes.io/projected/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-kube-api-access-9h2tp\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:42.278394 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.278168    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c20087fb-273e-4ba6-bc41-3a31cfeb30ec-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:42.578227 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.578191    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"]
Oct 30 22:08:42.583292 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:42.583257    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-nzzv2"]
Oct 30 22:08:43.056671 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:43.056637    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:43.449998 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:43.449965    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" path="/var/lib/kubelet/pods/c20087fb-273e-4ba6-bc41-3a31cfeb30ec/volumes"
Oct 30 22:08:45.434433 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:45.434394    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:47.886467 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:47.886433    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:48.362344 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:08:48.362308    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:08:52.802454 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:52.802411    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:08:57.945462 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:08:57.945421    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:01.555128 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:01.555038    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:01.764812 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:01.764760    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:03.087372 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:03.087336    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:03.330476 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:03.330439    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:03.487818 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:03.487780    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:08.523452 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:08.523413    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:13.612814 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:13.612774    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:16.000769 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:16.000726    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"]
Oct 30 22:09:16.110728 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:16.110692    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"]
Oct 30 22:09:16.377742 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:16.377634    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:16.531192 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:16.531146    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:17.929144 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:17.929116    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:09:17.932744 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:17.932722    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:09:17.934301 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:17.934280    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:09:17.937970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:17.937949    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:09:18.016670 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.016635    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"]
Oct 30 22:09:18.016908 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.016883    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp" podUID="5a469237-ca99-4126-938b-bcbee6d9835f" containerName="maas-api" containerID="cri-o://874a063057efb52e3dcf20403b8524f04c1500bdae90c8bad641f14dd8e23c09" gracePeriod=30
Oct 30 22:09:18.154558 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.154508    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:18.445215 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.443215    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:18.475802 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.475036    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:09:18.477507 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:09:18.477476    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:09:18.557459 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.557422    5586 generic.go:356] "Generic (PLEG): container finished" podID="5a469237-ca99-4126-938b-bcbee6d9835f" containerID="874a063057efb52e3dcf20403b8524f04c1500bdae90c8bad641f14dd8e23c09" exitCode=0
Oct 30 22:09:18.557972 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.557738    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"
Oct 30 22:09:18.558226 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.557759    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp" event={"ID":"5a469237-ca99-4126-938b-bcbee6d9835f","Type":"ContainerDied","Data":"874a063057efb52e3dcf20403b8524f04c1500bdae90c8bad641f14dd8e23c09"}
Oct 30 22:09:18.558395 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.558379    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp" event={"ID":"5a469237-ca99-4126-938b-bcbee6d9835f","Type":"ContainerDied","Data":"61296f0b1f1e5b0e82e375d3543f944919c958ebd999c4674be00279dd681c69"}
Oct 30 22:09:18.558506 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.558494    5586 scope.go:122] "RemoveContainer" containerID="874a063057efb52e3dcf20403b8524f04c1500bdae90c8bad641f14dd8e23c09"
Oct 30 22:09:18.576470 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.575002    5586 scope.go:122] "RemoveContainer" containerID="874a063057efb52e3dcf20403b8524f04c1500bdae90c8bad641f14dd8e23c09"
Oct 30 22:09:18.576470 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:09:18.575437    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"874a063057efb52e3dcf20403b8524f04c1500bdae90c8bad641f14dd8e23c09\": container with ID starting with 874a063057efb52e3dcf20403b8524f04c1500bdae90c8bad641f14dd8e23c09 not found: ID does not exist" containerID="874a063057efb52e3dcf20403b8524f04c1500bdae90c8bad641f14dd8e23c09"
Oct 30 22:09:18.576470 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.575477    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874a063057efb52e3dcf20403b8524f04c1500bdae90c8bad641f14dd8e23c09"} err="failed to get container status \"874a063057efb52e3dcf20403b8524f04c1500bdae90c8bad641f14dd8e23c09\": rpc error: code = NotFound desc = could not find container \"874a063057efb52e3dcf20403b8524f04c1500bdae90c8bad641f14dd8e23c09\": container with ID starting with 874a063057efb52e3dcf20403b8524f04c1500bdae90c8bad641f14dd8e23c09 not found: ID does not exist"
Oct 30 22:09:18.649088 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.648648    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a469237-ca99-4126-938b-bcbee6d9835f-kube-api-access-mth5p\" (UniqueName: \"kubernetes.io/projected/5a469237-ca99-4126-938b-bcbee6d9835f-kube-api-access-mth5p\") pod \"5a469237-ca99-4126-938b-bcbee6d9835f\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") "
Oct 30 22:09:18.649636 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.649609    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a469237-ca99-4126-938b-bcbee6d9835f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a469237-ca99-4126-938b-bcbee6d9835f-openshift-service-ca\") pod \"5a469237-ca99-4126-938b-bcbee6d9835f\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") "
Oct 30 22:09:18.660925 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.660829    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a469237-ca99-4126-938b-bcbee6d9835f-openshift-service-ca" pod "5a469237-ca99-4126-938b-bcbee6d9835f" (UID: "5a469237-ca99-4126-938b-bcbee6d9835f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:18.661152 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.661130    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-maas-api-tls\") pod \"5a469237-ca99-4126-938b-bcbee6d9835f\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") "
Oct 30 22:09:18.661354 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.661336    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-metrics-certs\") pod \"5a469237-ca99-4126-938b-bcbee6d9835f\" (UID: \"5a469237-ca99-4126-938b-bcbee6d9835f\") "
Oct 30 22:09:18.663770 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.663743    5586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a469237-ca99-4126-938b-bcbee6d9835f-openshift-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:18.666298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.665943    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-maas-api-tls" pod "5a469237-ca99-4126-938b-bcbee6d9835f" (UID: "5a469237-ca99-4126-938b-bcbee6d9835f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:09:18.667048 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.667020    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-metrics-certs" pod "5a469237-ca99-4126-938b-bcbee6d9835f" (UID: "5a469237-ca99-4126-938b-bcbee6d9835f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:09:18.667285 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.667220    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a469237-ca99-4126-938b-bcbee6d9835f-kube-api-access-mth5p" pod "5a469237-ca99-4126-938b-bcbee6d9835f" (UID: "5a469237-ca99-4126-938b-bcbee6d9835f"). InnerVolumeSpecName "kube-api-access-mth5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:18.765770 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.765687    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-maas-api-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:18.765770 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.765720    5586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a469237-ca99-4126-938b-bcbee6d9835f-metrics-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:18.765770 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.765730    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mth5p\" (UniqueName: \"kubernetes.io/projected/5a469237-ca99-4126-938b-bcbee6d9835f-kube-api-access-mth5p\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:18.996256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:18.995204    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"]
Oct 30 22:09:19.026317 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:19.022840    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-99f69c-877668cc4-cw5sp"]
Oct 30 22:09:19.455762 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:19.455724    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a469237-ca99-4126-938b-bcbee6d9835f" path="/var/lib/kubelet/pods/5a469237-ca99-4126-938b-bcbee6d9835f/volumes"
Oct 30 22:09:19.743535 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:19.743430    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:20.077622 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:20.077534    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:20.247812 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:20.247775    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:20.878596 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:20.878559    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:21.001404 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.001360    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l" podUID="dfeb8098-593b-437f-acb7-74348724dea2" containerName="payload-processing" containerID="cri-o://16cac7c0e3bd8f1d511e48b1f17946ed62456efd88278ff89f5e6d28de626bd2" gracePeriod=25
Oct 30 22:09:21.111947 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.111902    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd" podUID="b73ddad4-9be5-4fa2-9691-013116c525f7" containerName="payload-pre-processing" containerID="cri-o://8c4dd6c4878de9e9635af49ea4f2eae9073c81c1a664df0765b742be1c99620f" gracePeriod=25
Oct 30 22:09:21.377132 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.376714    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"
Oct 30 22:09:21.410287 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.410184    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dfeb8098-593b-437f-acb7-74348724dea2-kube-api-access-jssbx\" (UniqueName: \"kubernetes.io/projected/dfeb8098-593b-437f-acb7-74348724dea2-kube-api-access-jssbx\") pod \"dfeb8098-593b-437f-acb7-74348724dea2\" (UID: \"dfeb8098-593b-437f-acb7-74348724dea2\") "
Oct 30 22:09:21.410482 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.410304    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dfeb8098-593b-437f-acb7-74348724dea2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfeb8098-593b-437f-acb7-74348724dea2-plugins-config-volume\") pod \"dfeb8098-593b-437f-acb7-74348724dea2\" (UID: \"dfeb8098-593b-437f-acb7-74348724dea2\") "
Oct 30 22:09:21.410956 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.410929    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfeb8098-593b-437f-acb7-74348724dea2-plugins-config-volume" pod "dfeb8098-593b-437f-acb7-74348724dea2" (UID: "dfeb8098-593b-437f-acb7-74348724dea2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:21.411089 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.411039    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfeb8098-593b-437f-acb7-74348724dea2-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:21.415101 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.415063    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfeb8098-593b-437f-acb7-74348724dea2-kube-api-access-jssbx" pod "dfeb8098-593b-437f-acb7-74348724dea2" (UID: "dfeb8098-593b-437f-acb7-74348724dea2"). InnerVolumeSpecName "kube-api-access-jssbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:21.471694 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.471670    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"
Oct 30 22:09:21.511600 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.511566    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b73ddad4-9be5-4fa2-9691-013116c525f7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b73ddad4-9be5-4fa2-9691-013116c525f7-plugins-config-volume\") pod \"b73ddad4-9be5-4fa2-9691-013116c525f7\" (UID: \"b73ddad4-9be5-4fa2-9691-013116c525f7\") "
Oct 30 22:09:21.511812 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.511714    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b73ddad4-9be5-4fa2-9691-013116c525f7-kube-api-access-4n84r\" (UniqueName: \"kubernetes.io/projected/b73ddad4-9be5-4fa2-9691-013116c525f7-kube-api-access-4n84r\") pod \"b73ddad4-9be5-4fa2-9691-013116c525f7\" (UID: \"b73ddad4-9be5-4fa2-9691-013116c525f7\") "
Oct 30 22:09:21.511963 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.511939    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b73ddad4-9be5-4fa2-9691-013116c525f7-plugins-config-volume" pod "b73ddad4-9be5-4fa2-9691-013116c525f7" (UID: "b73ddad4-9be5-4fa2-9691-013116c525f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:21.512904 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.512876    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jssbx\" (UniqueName: \"kubernetes.io/projected/dfeb8098-593b-437f-acb7-74348724dea2-kube-api-access-jssbx\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:21.512904 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.512905    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b73ddad4-9be5-4fa2-9691-013116c525f7-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:21.514305 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.514283    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b73ddad4-9be5-4fa2-9691-013116c525f7-kube-api-access-4n84r" pod "b73ddad4-9be5-4fa2-9691-013116c525f7" (UID: "b73ddad4-9be5-4fa2-9691-013116c525f7"). InnerVolumeSpecName "kube-api-access-4n84r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:21.585571 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.585525    5586 generic.go:356] "Generic (PLEG): container finished" podID="b73ddad4-9be5-4fa2-9691-013116c525f7" containerID="8c4dd6c4878de9e9635af49ea4f2eae9073c81c1a664df0765b742be1c99620f" exitCode=0
Oct 30 22:09:21.585775 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.585581    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"
Oct 30 22:09:21.585775 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.585597    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd" event={"ID":"b73ddad4-9be5-4fa2-9691-013116c525f7","Type":"ContainerDied","Data":"8c4dd6c4878de9e9635af49ea4f2eae9073c81c1a664df0765b742be1c99620f"}
Oct 30 22:09:21.585775 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.585658    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd" event={"ID":"b73ddad4-9be5-4fa2-9691-013116c525f7","Type":"ContainerDied","Data":"d14a7cc01388f8640c273da5f693cf20cb0130945788727d61ce7343761c8fe3"}
Oct 30 22:09:21.585775 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.585686    5586 scope.go:122] "RemoveContainer" containerID="8c4dd6c4878de9e9635af49ea4f2eae9073c81c1a664df0765b742be1c99620f"
Oct 30 22:09:21.587880 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.587850    5586 generic.go:356] "Generic (PLEG): container finished" podID="dfeb8098-593b-437f-acb7-74348724dea2" containerID="16cac7c0e3bd8f1d511e48b1f17946ed62456efd88278ff89f5e6d28de626bd2" exitCode=0
Oct 30 22:09:21.588013 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.587892    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"
Oct 30 22:09:21.588013 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.587906    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l" event={"ID":"dfeb8098-593b-437f-acb7-74348724dea2","Type":"ContainerDied","Data":"16cac7c0e3bd8f1d511e48b1f17946ed62456efd88278ff89f5e6d28de626bd2"}
Oct 30 22:09:21.588013 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.587947    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l" event={"ID":"dfeb8098-593b-437f-acb7-74348724dea2","Type":"ContainerDied","Data":"c12dfeaf708ecaf152ded46bbc1c4212796d2c180975e8cba86f860157275e41"}
Oct 30 22:09:21.601801 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.601775    5586 scope.go:122] "RemoveContainer" containerID="8c4dd6c4878de9e9635af49ea4f2eae9073c81c1a664df0765b742be1c99620f"
Oct 30 22:09:21.602309 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:09:21.602278    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c4dd6c4878de9e9635af49ea4f2eae9073c81c1a664df0765b742be1c99620f\": container with ID starting with 8c4dd6c4878de9e9635af49ea4f2eae9073c81c1a664df0765b742be1c99620f not found: ID does not exist" containerID="8c4dd6c4878de9e9635af49ea4f2eae9073c81c1a664df0765b742be1c99620f"
Oct 30 22:09:21.602424 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.602320    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c4dd6c4878de9e9635af49ea4f2eae9073c81c1a664df0765b742be1c99620f"} err="failed to get container status \"8c4dd6c4878de9e9635af49ea4f2eae9073c81c1a664df0765b742be1c99620f\": rpc error: code = NotFound desc = could not find container \"8c4dd6c4878de9e9635af49ea4f2eae9073c81c1a664df0765b742be1c99620f\": container with ID starting with 8c4dd6c4878de9e9635af49ea4f2eae9073c81c1a664df0765b742be1c99620f not found: ID does not exist"
Oct 30 22:09:21.602424 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.602345    5586 scope.go:122] "RemoveContainer" containerID="16cac7c0e3bd8f1d511e48b1f17946ed62456efd88278ff89f5e6d28de626bd2"
Oct 30 22:09:21.612900 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.612874    5586 scope.go:122] "RemoveContainer" containerID="16cac7c0e3bd8f1d511e48b1f17946ed62456efd88278ff89f5e6d28de626bd2"
Oct 30 22:09:21.614041 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:09:21.613650    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16cac7c0e3bd8f1d511e48b1f17946ed62456efd88278ff89f5e6d28de626bd2\": container with ID starting with 16cac7c0e3bd8f1d511e48b1f17946ed62456efd88278ff89f5e6d28de626bd2 not found: ID does not exist" containerID="16cac7c0e3bd8f1d511e48b1f17946ed62456efd88278ff89f5e6d28de626bd2"
Oct 30 22:09:21.614041 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.613692    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16cac7c0e3bd8f1d511e48b1f17946ed62456efd88278ff89f5e6d28de626bd2"} err="failed to get container status \"16cac7c0e3bd8f1d511e48b1f17946ed62456efd88278ff89f5e6d28de626bd2\": rpc error: code = NotFound desc = could not find container \"16cac7c0e3bd8f1d511e48b1f17946ed62456efd88278ff89f5e6d28de626bd2\": container with ID starting with 16cac7c0e3bd8f1d511e48b1f17946ed62456efd88278ff89f5e6d28de626bd2 not found: ID does not exist"
Oct 30 22:09:21.614223 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.614203    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n84r\" (UniqueName: \"kubernetes.io/projected/b73ddad4-9be5-4fa2-9691-013116c525f7-kube-api-access-4n84r\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:21.751079 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.750969    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"]
Oct 30 22:09:21.754469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.754432    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-ea3cb2-675f44bd99-9dbgd"]
Oct 30 22:09:21.859735 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.859691    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"]
Oct 30 22:09:21.863970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:21.863943    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ea3cb2-688fb7cdb9-kxp5l"]
Oct 30 22:09:23.456035 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:23.456003    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b73ddad4-9be5-4fa2-9691-013116c525f7" path="/var/lib/kubelet/pods/b73ddad4-9be5-4fa2-9691-013116c525f7/volumes"
Oct 30 22:09:23.456538 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:23.456521    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dfeb8098-593b-437f-acb7-74348724dea2" path="/var/lib/kubelet/pods/dfeb8098-593b-437f-acb7-74348724dea2/volumes"
Oct 30 22:09:25.090830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:25.090797    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:26.895707 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:26.895669    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:27.092490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:27.092443    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:27.285442 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:27.285407    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"]
Oct 30 22:09:27.285712 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:27.285673    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm" podUID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerName="istio-proxy" containerID="cri-o://7447c2eb2d2afb6bc5c6e3a4cc34a9950fac8038474543d1a440a6098c7e9f48" gracePeriod=30
Oct 30 22:09:27.335858 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:27.335822    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:27.581704 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:27.581391    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:28.038307 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:28.037052    5586 patch_prober.go:29] interesting pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": read tcp 10.134.0.2:35510->10.134.0.137:15021: read: connection reset by peer" start-of-body=
Oct 30 22:09:28.038307 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:28.037155    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb" podUID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": read tcp 10.134.0.2:35510->10.134.0.137:15021: read: connection reset by peer"
Oct 30 22:09:28.647162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:28.647130    5586 generic.go:356] "Generic (PLEG): container finished" podID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerID="e0b7d75fd324d899b6843219845204ebdf3e2331f7623f90d1078d603337631d" exitCode=0
Oct 30 22:09:28.647384 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:28.647210    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb" event={"ID":"aae92a23-65e7-4f07-bbf8-934cc18e916e","Type":"ContainerDied","Data":"e0b7d75fd324d899b6843219845204ebdf3e2331f7623f90d1078d603337631d"}
Oct 30 22:09:28.647719 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:28.647704    5586 scope.go:122] "RemoveContainer" containerID="e0b7d75fd324d899b6843219845204ebdf3e2331f7623f90d1078d603337631d"
Oct 30 22:09:28.649482 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:28.649448    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:09:28.649558 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:28.649536    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:09:28.649603 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:28.649568    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:09:29.656730 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:29.656691    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb" event={"ID":"aae92a23-65e7-4f07-bbf8-934cc18e916e","Type":"ContainerStarted","Data":"8d103da0d07c17ada9f15c90d901c2beabf81a9d856f7d1894001544b7fc1814"}
Oct 30 22:09:30.174586 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:30.174538    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:30.453898 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:30.453806    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:09:30.454095 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:30.454069    5586 patch_prober.go:29] interesting pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:09:30.454167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:30.454129    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb" podUID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 22:09:31.454694 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:31.454649    5586 patch_prober.go:29] interesting pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:09:31.455106 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:31.454718    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb" podUID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 22:09:32.454247 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.454187    5586 patch_prober.go:29] interesting pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:09:32.454444 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.454274    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb" podUID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 22:09:32.616115 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.616083    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:09:32.680641 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.680609    5586 generic.go:356] "Generic (PLEG): container finished" podID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerID="7447c2eb2d2afb6bc5c6e3a4cc34a9950fac8038474543d1a440a6098c7e9f48" exitCode=0
Oct 30 22:09:32.680819 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.680661    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"
Oct 30 22:09:32.680819 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.680676    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm" event={"ID":"a76b490b-c349-4a5c-ba7e-6445596e7ee0","Type":"ContainerDied","Data":"7447c2eb2d2afb6bc5c6e3a4cc34a9950fac8038474543d1a440a6098c7e9f48"}
Oct 30 22:09:32.680819 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.680719    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm" event={"ID":"a76b490b-c349-4a5c-ba7e-6445596e7ee0","Type":"ContainerDied","Data":"4d62ce38a71e86f333494ae09c284bd29908ee5e674ef7113f8e79b13799916d"}
Oct 30 22:09:32.680819 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.680741    5586 scope.go:122] "RemoveContainer" containerID="7447c2eb2d2afb6bc5c6e3a4cc34a9950fac8038474543d1a440a6098c7e9f48"
Oct 30 22:09:32.692185 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.692160    5586 scope.go:122] "RemoveContainer" containerID="e2373d3d6e42c85d745de5944cf1c7f7290636749099a83030a7a2c142bc01db"
Oct 30 22:09:32.702007 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.701985    5586 scope.go:122] "RemoveContainer" containerID="7447c2eb2d2afb6bc5c6e3a4cc34a9950fac8038474543d1a440a6098c7e9f48"
Oct 30 22:09:32.702335 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:09:32.702314    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7447c2eb2d2afb6bc5c6e3a4cc34a9950fac8038474543d1a440a6098c7e9f48\": container with ID starting with 7447c2eb2d2afb6bc5c6e3a4cc34a9950fac8038474543d1a440a6098c7e9f48 not found: ID does not exist" containerID="7447c2eb2d2afb6bc5c6e3a4cc34a9950fac8038474543d1a440a6098c7e9f48"
Oct 30 22:09:32.702440 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.702348    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7447c2eb2d2afb6bc5c6e3a4cc34a9950fac8038474543d1a440a6098c7e9f48"} err="failed to get container status \"7447c2eb2d2afb6bc5c6e3a4cc34a9950fac8038474543d1a440a6098c7e9f48\": rpc error: code = NotFound desc = could not find container \"7447c2eb2d2afb6bc5c6e3a4cc34a9950fac8038474543d1a440a6098c7e9f48\": container with ID starting with 7447c2eb2d2afb6bc5c6e3a4cc34a9950fac8038474543d1a440a6098c7e9f48 not found: ID does not exist"
Oct 30 22:09:32.702440 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.702378    5586 scope.go:122] "RemoveContainer" containerID="e2373d3d6e42c85d745de5944cf1c7f7290636749099a83030a7a2c142bc01db"
Oct 30 22:09:32.702655 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:09:32.702635    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2373d3d6e42c85d745de5944cf1c7f7290636749099a83030a7a2c142bc01db\": container with ID starting with e2373d3d6e42c85d745de5944cf1c7f7290636749099a83030a7a2c142bc01db not found: ID does not exist" containerID="e2373d3d6e42c85d745de5944cf1c7f7290636749099a83030a7a2c142bc01db"
Oct 30 22:09:32.702706 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.702676    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2373d3d6e42c85d745de5944cf1c7f7290636749099a83030a7a2c142bc01db"} err="failed to get container status \"e2373d3d6e42c85d745de5944cf1c7f7290636749099a83030a7a2c142bc01db\": rpc error: code = NotFound desc = could not find container \"e2373d3d6e42c85d745de5944cf1c7f7290636749099a83030a7a2c142bc01db\": container with ID starting with e2373d3d6e42c85d745de5944cf1c7f7290636749099a83030a7a2c142bc01db not found: ID does not exist"
Oct 30 22:09:32.809808 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.809762    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a76b490b-c349-4a5c-ba7e-6445596e7ee0-kube-api-access-6mm6z\" (UniqueName: \"kubernetes.io/projected/a76b490b-c349-4a5c-ba7e-6445596e7ee0-kube-api-access-6mm6z\") pod \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") "
Oct 30 22:09:32.809989 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.809877    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-workload-certs\") pod \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") "
Oct 30 22:09:32.809989 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.809975    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-credential-socket\") pod \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") "
Oct 30 22:09:32.810137 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.810035    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-data\") pod \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") "
Oct 30 22:09:32.810137 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.810097    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-workload-socket\") pod \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") "
Oct 30 22:09:32.810280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.810135    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-token\" (UniqueName: \"kubernetes.io/projected/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-token\") pod \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") "
Oct 30 22:09:32.810280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.810183    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-envoy\") pod \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") "
Oct 30 22:09:32.810280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.810184    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-workload-certs" pod "a76b490b-c349-4a5c-ba7e-6445596e7ee0" (UID: "a76b490b-c349-4a5c-ba7e-6445596e7ee0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:32.810280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.810268    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-podinfo\") pod \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") "
Oct 30 22:09:32.810486 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.810359    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istiod-ca-cert\") pod \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\" (UID: \"a76b490b-c349-4a5c-ba7e-6445596e7ee0\") "
Oct 30 22:09:32.810486 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.810446    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-data" pod "a76b490b-c349-4a5c-ba7e-6445596e7ee0" (UID: "a76b490b-c349-4a5c-ba7e-6445596e7ee0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:32.810598 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.810487    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-credential-socket" pod "a76b490b-c349-4a5c-ba7e-6445596e7ee0" (UID: "a76b490b-c349-4a5c-ba7e-6445596e7ee0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:32.810741 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.810719    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-workload-socket" pod "a76b490b-c349-4a5c-ba7e-6445596e7ee0" (UID: "a76b490b-c349-4a5c-ba7e-6445596e7ee0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:32.811035 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.811017    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istiod-ca-cert" pod "a76b490b-c349-4a5c-ba7e-6445596e7ee0" (UID: "a76b490b-c349-4a5c-ba7e-6445596e7ee0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:32.811140 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.811093    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.811140 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.811116    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.811140 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.811130    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.811307 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.811143    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.812966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.812937    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a76b490b-c349-4a5c-ba7e-6445596e7ee0-kube-api-access-6mm6z" pod "a76b490b-c349-4a5c-ba7e-6445596e7ee0" (UID: "a76b490b-c349-4a5c-ba7e-6445596e7ee0"). InnerVolumeSpecName "kube-api-access-6mm6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:32.813064 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.813031    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-envoy" pod "a76b490b-c349-4a5c-ba7e-6445596e7ee0" (UID: "a76b490b-c349-4a5c-ba7e-6445596e7ee0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:32.814196 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.814170    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-podinfo" pod "a76b490b-c349-4a5c-ba7e-6445596e7ee0" (UID: "a76b490b-c349-4a5c-ba7e-6445596e7ee0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:09:32.814548 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.814522    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-token" pod "a76b490b-c349-4a5c-ba7e-6445596e7ee0" (UID: "a76b490b-c349-4a5c-ba7e-6445596e7ee0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:32.912815 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.912778    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.912815 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.912824    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.913026 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.912841    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.913026 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.912854    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a76b490b-c349-4a5c-ba7e-6445596e7ee0-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:32.913026 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:32.912867    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mm6z\" (UniqueName: \"kubernetes.io/projected/a76b490b-c349-4a5c-ba7e-6445596e7ee0-kube-api-access-6mm6z\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:33.105799 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:33.105705    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"]
Oct 30 22:09:33.109140 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:33.109113    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-99f69c-openshift-default-8f7ff99dc-j4vpm"]
Oct 30 22:09:33.454282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:33.454216    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" path="/var/lib/kubelet/pods/a76b490b-c349-4a5c-ba7e-6445596e7ee0/volumes"
Oct 30 22:09:33.454544 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:33.454521    5586 patch_prober.go:29] interesting pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:09:33.454618 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:33.454568    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb" podUID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 22:09:34.453742 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:34.453692    5586 patch_prober.go:29] interesting pod/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:09:34.454146 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:34.453763    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb" podUID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 22:09:35.453558 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:35.453517    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:35.463843 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:35.463807    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:09:35.465555 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:35.465515    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:09:35.467395 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:35.467373    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:09:39.009887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:39.009662    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:39.187329 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:39.187285    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:40.635330 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:40.635293    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:45.814799 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:45.814761    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:48.363258 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:09:48.363204    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:09:50.941744 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:50.941700    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:51.095363 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:51.095320    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:09:56.285269 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:09:56.285208    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:01.577952 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:01.577914    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:05.648103 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:05.648067    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"]
Oct 30 22:10:05.648547 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:05.648276    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q" podUID="5fca9942-5704-4346-8c4f-abe2c9bfefe2" containerName="maas-api" containerID="cri-o://d32a0a1ecfa5f29b4095484466949a85f94865d655a3ac6eb58f4dd9c15ac22e" gracePeriod=30
Oct 30 22:10:05.776776 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:05.776733    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:05.944315 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:05.944284    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:10:05.948706 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:05.948674    5586 generic.go:356] "Generic (PLEG): container finished" podID="5fca9942-5704-4346-8c4f-abe2c9bfefe2" containerID="d32a0a1ecfa5f29b4095484466949a85f94865d655a3ac6eb58f4dd9c15ac22e" exitCode=0
Oct 30 22:10:05.948854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:05.948720    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"
Oct 30 22:10:05.948854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:05.948766    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q" event={"ID":"5fca9942-5704-4346-8c4f-abe2c9bfefe2","Type":"ContainerDied","Data":"d32a0a1ecfa5f29b4095484466949a85f94865d655a3ac6eb58f4dd9c15ac22e"}
Oct 30 22:10:05.948854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:05.948822    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q" event={"ID":"5fca9942-5704-4346-8c4f-abe2c9bfefe2","Type":"ContainerDied","Data":"31553e6df30c943c5b8355e98d3d41460f3414b4c9a76a4c663cf17f219e60e3"}
Oct 30 22:10:05.948854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:05.948850    5586 scope.go:122] "RemoveContainer" containerID="d32a0a1ecfa5f29b4095484466949a85f94865d655a3ac6eb58f4dd9c15ac22e"
Oct 30 22:10:05.959377 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:05.959352    5586 scope.go:122] "RemoveContainer" containerID="d32a0a1ecfa5f29b4095484466949a85f94865d655a3ac6eb58f4dd9c15ac22e"
Oct 30 22:10:05.959666 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:10:05.959647    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d32a0a1ecfa5f29b4095484466949a85f94865d655a3ac6eb58f4dd9c15ac22e\": container with ID starting with d32a0a1ecfa5f29b4095484466949a85f94865d655a3ac6eb58f4dd9c15ac22e not found: ID does not exist" containerID="d32a0a1ecfa5f29b4095484466949a85f94865d655a3ac6eb58f4dd9c15ac22e"
Oct 30 22:10:05.959717 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:05.959675    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d32a0a1ecfa5f29b4095484466949a85f94865d655a3ac6eb58f4dd9c15ac22e"} err="failed to get container status \"d32a0a1ecfa5f29b4095484466949a85f94865d655a3ac6eb58f4dd9c15ac22e\": rpc error: code = NotFound desc = could not find container \"d32a0a1ecfa5f29b4095484466949a85f94865d655a3ac6eb58f4dd9c15ac22e\": container with ID starting with d32a0a1ecfa5f29b4095484466949a85f94865d655a3ac6eb58f4dd9c15ac22e not found: ID does not exist"
Oct 30 22:10:06.046624 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:06.046166    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5fca9942-5704-4346-8c4f-abe2c9bfefe2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fca9942-5704-4346-8c4f-abe2c9bfefe2-openshift-service-ca\") pod \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") "
Oct 30 22:10:06.046942 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:06.046598    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fca9942-5704-4346-8c4f-abe2c9bfefe2-openshift-service-ca" pod "5fca9942-5704-4346-8c4f-abe2c9bfefe2" (UID: "5fca9942-5704-4346-8c4f-abe2c9bfefe2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:06.047167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:06.047147    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-maas-api-tls\") pod \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") "
Oct 30 22:10:06.047854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:06.047829    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fca9942-5704-4346-8c4f-abe2c9bfefe2-kube-api-access-wt9nb\" (UniqueName: \"kubernetes.io/projected/5fca9942-5704-4346-8c4f-abe2c9bfefe2-kube-api-access-wt9nb\") pod \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") "
Oct 30 22:10:06.048730 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:06.048406    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-metrics-certs\") pod \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\" (UID: \"5fca9942-5704-4346-8c4f-abe2c9bfefe2\") "
Oct 30 22:10:06.049557 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:06.049521    5586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fca9942-5704-4346-8c4f-abe2c9bfefe2-openshift-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:06.060342 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:06.057683    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-maas-api-tls" pod "5fca9942-5704-4346-8c4f-abe2c9bfefe2" (UID: "5fca9942-5704-4346-8c4f-abe2c9bfefe2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:06.077098 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:06.077058    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-metrics-certs" pod "5fca9942-5704-4346-8c4f-abe2c9bfefe2" (UID: "5fca9942-5704-4346-8c4f-abe2c9bfefe2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:06.077578 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:06.077550    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fca9942-5704-4346-8c4f-abe2c9bfefe2-kube-api-access-wt9nb" pod "5fca9942-5704-4346-8c4f-abe2c9bfefe2" (UID: "5fca9942-5704-4346-8c4f-abe2c9bfefe2"). InnerVolumeSpecName "kube-api-access-wt9nb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:06.150646 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:06.150613    5586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-metrics-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:06.150646 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:06.150647    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fca9942-5704-4346-8c4f-abe2c9bfefe2-maas-api-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:06.150866 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:06.150666    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wt9nb\" (UniqueName: \"kubernetes.io/projected/5fca9942-5704-4346-8c4f-abe2c9bfefe2-kube-api-access-wt9nb\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:06.369829 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:06.369786    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"]
Oct 30 22:10:06.384116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:06.377366    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-ea3cb2-6b7c8897c8-csv8q"]
Oct 30 22:10:07.298680 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:07.298638    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:07.451116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:07.451081    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5fca9942-5704-4346-8c4f-abe2c9bfefe2" path="/var/lib/kubelet/pods/5fca9942-5704-4346-8c4f-abe2c9bfefe2/volumes"
Oct 30 22:10:07.518783 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:07.518740    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:07.737010 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:07.736965    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:08.269898 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:08.269864    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:09.964487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:09.964452    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:10.172790 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:10.172756    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:12.218077 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:12.218037    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:13.351386 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.351342    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"]
Oct 30 22:10:13.351859 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.351569    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc" podUID="1ae3b993-2c09-41f3-bcc7-eec79cae6198" containerName="maas-api" containerID="cri-o://a16d46e69f8d20d7c2144a47ed2acc5582b4d8ca1908f0c2940408aeef0daf0e" gracePeriod=30
Oct 30 22:10:13.480754 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.480715    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:13.622584 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.622519    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:10:13.789112 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.777710    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ae3b993-2c09-41f3-bcc7-eec79cae6198-kube-api-access-8mlqz\" (UniqueName: \"kubernetes.io/projected/1ae3b993-2c09-41f3-bcc7-eec79cae6198-kube-api-access-8mlqz\") pod \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") "
Oct 30 22:10:13.789642 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.789605    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-maas-api-tls\") pod \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") "
Oct 30 22:10:13.790053 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.790033    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ae3b993-2c09-41f3-bcc7-eec79cae6198-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae3b993-2c09-41f3-bcc7-eec79cae6198-openshift-service-ca\") pod \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") "
Oct 30 22:10:13.790372 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.790353    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-metrics-certs\") pod \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\" (UID: \"1ae3b993-2c09-41f3-bcc7-eec79cae6198\") "
Oct 30 22:10:13.792446 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.792405    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ae3b993-2c09-41f3-bcc7-eec79cae6198-openshift-service-ca" pod "1ae3b993-2c09-41f3-bcc7-eec79cae6198" (UID: "1ae3b993-2c09-41f3-bcc7-eec79cae6198"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:13.795226 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.795198    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-maas-api-tls" pod "1ae3b993-2c09-41f3-bcc7-eec79cae6198" (UID: "1ae3b993-2c09-41f3-bcc7-eec79cae6198"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:13.818853 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.799104    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ae3b993-2c09-41f3-bcc7-eec79cae6198-kube-api-access-8mlqz" pod "1ae3b993-2c09-41f3-bcc7-eec79cae6198" (UID: "1ae3b993-2c09-41f3-bcc7-eec79cae6198"). InnerVolumeSpecName "kube-api-access-8mlqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:13.822971 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.822935    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-metrics-certs" pod "1ae3b993-2c09-41f3-bcc7-eec79cae6198" (UID: "1ae3b993-2c09-41f3-bcc7-eec79cae6198"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:13.845468 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.845428    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:13.895503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.894184    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-maas-api-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:13.895503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.894224    5586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae3b993-2c09-41f3-bcc7-eec79cae6198-openshift-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:13.895503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.894258    5586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ae3b993-2c09-41f3-bcc7-eec79cae6198-metrics-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:13.895503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:13.894277    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mlqz\" (UniqueName: \"kubernetes.io/projected/1ae3b993-2c09-41f3-bcc7-eec79cae6198-kube-api-access-8mlqz\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:14.014346 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:14.014304    5586 generic.go:356] "Generic (PLEG): container finished" podID="1ae3b993-2c09-41f3-bcc7-eec79cae6198" containerID="a16d46e69f8d20d7c2144a47ed2acc5582b4d8ca1908f0c2940408aeef0daf0e" exitCode=0
Oct 30 22:10:14.014564 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:14.014365    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"
Oct 30 22:10:14.014564 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:14.014385    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc" event={"ID":"1ae3b993-2c09-41f3-bcc7-eec79cae6198","Type":"ContainerDied","Data":"a16d46e69f8d20d7c2144a47ed2acc5582b4d8ca1908f0c2940408aeef0daf0e"}
Oct 30 22:10:14.014564 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:14.014443    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc" event={"ID":"1ae3b993-2c09-41f3-bcc7-eec79cae6198","Type":"ContainerDied","Data":"7c872e1032e4d442e00abb4e530460cbb00889470aa7252795ee76603c4242b2"}
Oct 30 22:10:14.014564 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:14.014471    5586 scope.go:122] "RemoveContainer" containerID="a16d46e69f8d20d7c2144a47ed2acc5582b4d8ca1908f0c2940408aeef0daf0e"
Oct 30 22:10:14.026639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:14.026618    5586 scope.go:122] "RemoveContainer" containerID="a16d46e69f8d20d7c2144a47ed2acc5582b4d8ca1908f0c2940408aeef0daf0e"
Oct 30 22:10:14.026965 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:10:14.026942    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a16d46e69f8d20d7c2144a47ed2acc5582b4d8ca1908f0c2940408aeef0daf0e\": container with ID starting with a16d46e69f8d20d7c2144a47ed2acc5582b4d8ca1908f0c2940408aeef0daf0e not found: ID does not exist" containerID="a16d46e69f8d20d7c2144a47ed2acc5582b4d8ca1908f0c2940408aeef0daf0e"
Oct 30 22:10:14.027019 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:14.026974    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a16d46e69f8d20d7c2144a47ed2acc5582b4d8ca1908f0c2940408aeef0daf0e"} err="failed to get container status \"a16d46e69f8d20d7c2144a47ed2acc5582b4d8ca1908f0c2940408aeef0daf0e\": rpc error: code = NotFound desc = could not find container \"a16d46e69f8d20d7c2144a47ed2acc5582b4d8ca1908f0c2940408aeef0daf0e\": container with ID starting with a16d46e69f8d20d7c2144a47ed2acc5582b4d8ca1908f0c2940408aeef0daf0e not found: ID does not exist"
Oct 30 22:10:14.175703 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:14.175667    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:14.178463 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:14.178421    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"]
Oct 30 22:10:14.184023 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:14.184000    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9d4043-776c45bdd6-wf5nc"]
Oct 30 22:10:14.902538 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:14.902497    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:15.104928 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:15.104891    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:15.396875 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:15.396833    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"]
Oct 30 22:10:15.397071 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:15.397049    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerName="istio-proxy" containerID="cri-o://00b90ec64c2d470a33e2c529e974894b70a8791c1f24ad975b7138c7dc5bd1b9" gracePeriod=30
Oct 30 22:10:15.398022 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:15.398000    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:15.449581 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:15.449550    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ae3b993-2c09-41f3-bcc7-eec79cae6198" path="/var/lib/kubelet/pods/1ae3b993-2c09-41f3-bcc7-eec79cae6198/volumes"
Oct 30 22:10:15.565193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:15.565154    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:16.136407 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:16.136370    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:18.362902 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:10:18.362867    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:10:19.705946 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:19.705909    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:19.958327 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:19.958200    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:20.394555 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.394517    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:20.675658 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.675629    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:10:20.696932 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.696891    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istiod-ca-cert\") pod \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") "
Oct 30 22:10:20.697128 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.696954    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-workload-socket\") pod \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") "
Oct 30 22:10:20.697128 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.697033    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-envoy\") pod \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") "
Oct 30 22:10:20.697128 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.697097    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-token\" (UniqueName: \"kubernetes.io/projected/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-token\") pod \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") "
Oct 30 22:10:20.697347 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.697136    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-workload-certs\") pod \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") "
Oct 30 22:10:20.697347 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.697255    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1307e68-d9b4-48e0-bad6-44854d56a0c4-kube-api-access-tqpjq\" (UniqueName: \"kubernetes.io/projected/e1307e68-d9b4-48e0-bad6-44854d56a0c4-kube-api-access-tqpjq\") pod \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") "
Oct 30 22:10:20.697347 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.697264    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-workload-socket" pod "e1307e68-d9b4-48e0-bad6-44854d56a0c4" (UID: "e1307e68-d9b4-48e0-bad6-44854d56a0c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:20.697347 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.697290    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istiod-ca-cert" pod "e1307e68-d9b4-48e0-bad6-44854d56a0c4" (UID: "e1307e68-d9b4-48e0-bad6-44854d56a0c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:20.697347 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.697316    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-podinfo\") pod \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") "
Oct 30 22:10:20.697596 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.697405    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-data\") pod \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") "
Oct 30 22:10:20.697596 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.697515    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-credential-socket\") pod \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\" (UID: \"e1307e68-d9b4-48e0-bad6-44854d56a0c4\") "
Oct 30 22:10:20.697596 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.697541    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-workload-certs" pod "e1307e68-d9b4-48e0-bad6-44854d56a0c4" (UID: "e1307e68-d9b4-48e0-bad6-44854d56a0c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:20.697746 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.697681    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-data" pod "e1307e68-d9b4-48e0-bad6-44854d56a0c4" (UID: "e1307e68-d9b4-48e0-bad6-44854d56a0c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:20.697796 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.697762    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-credential-socket" pod "e1307e68-d9b4-48e0-bad6-44854d56a0c4" (UID: "e1307e68-d9b4-48e0-bad6-44854d56a0c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:20.698114 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.698092    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:20.698208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.698118    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:20.698208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.698132    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:20.698208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.698143    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:20.698208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.698158    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:20.699967 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.699937    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-podinfo" pod "e1307e68-d9b4-48e0-bad6-44854d56a0c4" (UID: "e1307e68-d9b4-48e0-bad6-44854d56a0c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:10:20.699967 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.699952    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-token" pod "e1307e68-d9b4-48e0-bad6-44854d56a0c4" (UID: "e1307e68-d9b4-48e0-bad6-44854d56a0c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:20.700107 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.699971    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-envoy" pod "e1307e68-d9b4-48e0-bad6-44854d56a0c4" (UID: "e1307e68-d9b4-48e0-bad6-44854d56a0c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:20.700107 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.699990    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1307e68-d9b4-48e0-bad6-44854d56a0c4-kube-api-access-tqpjq" pod "e1307e68-d9b4-48e0-bad6-44854d56a0c4" (UID: "e1307e68-d9b4-48e0-bad6-44854d56a0c4"). InnerVolumeSpecName "kube-api-access-tqpjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:20.799247 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.799191    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:20.799247 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.799224    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:20.799677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.799258    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1307e68-d9b4-48e0-bad6-44854d56a0c4-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:20.799677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:20.799268    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqpjq\" (UniqueName: \"kubernetes.io/projected/e1307e68-d9b4-48e0-bad6-44854d56a0c4-kube-api-access-tqpjq\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:21.075684 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.075647    5586 generic.go:356] "Generic (PLEG): container finished" podID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerID="00b90ec64c2d470a33e2c529e974894b70a8791c1f24ad975b7138c7dc5bd1b9" exitCode=0
Oct 30 22:10:21.075869 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.075705    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"
Oct 30 22:10:21.075869 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.075726    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" event={"ID":"e1307e68-d9b4-48e0-bad6-44854d56a0c4","Type":"ContainerDied","Data":"00b90ec64c2d470a33e2c529e974894b70a8791c1f24ad975b7138c7dc5bd1b9"}
Oct 30 22:10:21.075869 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.075826    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2" event={"ID":"e1307e68-d9b4-48e0-bad6-44854d56a0c4","Type":"ContainerDied","Data":"89312ea12fc42ab8fc4a84cb20eb3a00e75e102082f2302e143f74f0f2e1280f"}
Oct 30 22:10:21.075869 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.075862    5586 scope.go:122] "RemoveContainer" containerID="00b90ec64c2d470a33e2c529e974894b70a8791c1f24ad975b7138c7dc5bd1b9"
Oct 30 22:10:21.087901 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.087873    5586 scope.go:122] "RemoveContainer" containerID="190677165c88e636d400ced972849c243e1c36f772db8fde1702cb9a3caa88a7"
Oct 30 22:10:21.098079 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.098056    5586 scope.go:122] "RemoveContainer" containerID="00b90ec64c2d470a33e2c529e974894b70a8791c1f24ad975b7138c7dc5bd1b9"
Oct 30 22:10:21.098402 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:10:21.098381    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00b90ec64c2d470a33e2c529e974894b70a8791c1f24ad975b7138c7dc5bd1b9\": container with ID starting with 00b90ec64c2d470a33e2c529e974894b70a8791c1f24ad975b7138c7dc5bd1b9 not found: ID does not exist" containerID="00b90ec64c2d470a33e2c529e974894b70a8791c1f24ad975b7138c7dc5bd1b9"
Oct 30 22:10:21.098490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.098415    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00b90ec64c2d470a33e2c529e974894b70a8791c1f24ad975b7138c7dc5bd1b9"} err="failed to get container status \"00b90ec64c2d470a33e2c529e974894b70a8791c1f24ad975b7138c7dc5bd1b9\": rpc error: code = NotFound desc = could not find container \"00b90ec64c2d470a33e2c529e974894b70a8791c1f24ad975b7138c7dc5bd1b9\": container with ID starting with 00b90ec64c2d470a33e2c529e974894b70a8791c1f24ad975b7138c7dc5bd1b9 not found: ID does not exist"
Oct 30 22:10:21.098490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.098442    5586 scope.go:122] "RemoveContainer" containerID="190677165c88e636d400ced972849c243e1c36f772db8fde1702cb9a3caa88a7"
Oct 30 22:10:21.098735 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:10:21.098716    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"190677165c88e636d400ced972849c243e1c36f772db8fde1702cb9a3caa88a7\": container with ID starting with 190677165c88e636d400ced972849c243e1c36f772db8fde1702cb9a3caa88a7 not found: ID does not exist" containerID="190677165c88e636d400ced972849c243e1c36f772db8fde1702cb9a3caa88a7"
Oct 30 22:10:21.098793 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.098745    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"190677165c88e636d400ced972849c243e1c36f772db8fde1702cb9a3caa88a7"} err="failed to get container status \"190677165c88e636d400ced972849c243e1c36f772db8fde1702cb9a3caa88a7\": rpc error: code = NotFound desc = could not find container \"190677165c88e636d400ced972849c243e1c36f772db8fde1702cb9a3caa88a7\": container with ID starting with 190677165c88e636d400ced972849c243e1c36f772db8fde1702cb9a3caa88a7 not found: ID does not exist"
Oct 30 22:10:21.269370 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.269327    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"]
Oct 30 22:10:21.272646 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.272617    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ea3cb2-openshift-default-849dd5b97f-p62x2"]
Oct 30 22:10:21.449563 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.449530    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" path="/var/lib/kubelet/pods/e1307e68-d9b4-48e0-bad6-44854d56a0c4/volumes"
Oct 30 22:10:21.650796 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.650763    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:21.864075 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.863968    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:21.999769 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.999137    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"]
Oct 30 22:10:21.999769 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:21.999426    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb" podUID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerName="istio-proxy" containerID="cri-o://8d103da0d07c17ada9f15c90d901c2beabf81a9d856f7d1894001544b7fc1814" gracePeriod=30
Oct 30 22:10:22.042772 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:22.042737    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:23.190540 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:23.190499    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:23.346911 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:23.346861    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:23.546427 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:23.546331    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:24.274748 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:24.274712    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:24.923985 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:24.923946    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:25.091245 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:25.091193    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:25.828278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:25.828213    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:26.980199 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:26.980160    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:27.127695 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.127663    5586 generic.go:356] "Generic (PLEG): container finished" podID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerID="8d103da0d07c17ada9f15c90d901c2beabf81a9d856f7d1894001544b7fc1814" exitCode=0
Oct 30 22:10:27.127849 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.127744    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb" event={"ID":"aae92a23-65e7-4f07-bbf8-934cc18e916e","Type":"ContainerDied","Data":"8d103da0d07c17ada9f15c90d901c2beabf81a9d856f7d1894001544b7fc1814"}
Oct 30 22:10:27.127849 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.127811    5586 scope.go:122] "RemoveContainer" containerID="e0b7d75fd324d899b6843219845204ebdf3e2331f7623f90d1078d603337631d"
Oct 30 22:10:27.179849 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.179810    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:27.324499 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.324468    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:10:27.406881 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.406846    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-workload-certs\") pod \"aae92a23-65e7-4f07-bbf8-934cc18e916e\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") "
Oct 30 22:10:27.407053 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.407008    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-podinfo\") pod \"aae92a23-65e7-4f07-bbf8-934cc18e916e\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") "
Oct 30 22:10:27.407053 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.407039    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-data\") pod \"aae92a23-65e7-4f07-bbf8-934cc18e916e\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") "
Oct 30 22:10:27.407187 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.407065    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-envoy\") pod \"aae92a23-65e7-4f07-bbf8-934cc18e916e\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") "
Oct 30 22:10:27.407187 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.407138    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-workload-socket\") pod \"aae92a23-65e7-4f07-bbf8-934cc18e916e\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") "
Oct 30 22:10:27.407187 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.407159    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-workload-certs" pod "aae92a23-65e7-4f07-bbf8-934cc18e916e" (UID: "aae92a23-65e7-4f07-bbf8-934cc18e916e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:27.407406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.407211    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-credential-socket\") pod \"aae92a23-65e7-4f07-bbf8-934cc18e916e\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") "
Oct 30 22:10:27.407406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.407379    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aae92a23-65e7-4f07-bbf8-934cc18e916e-kube-api-access-bq8t6\" (UniqueName: \"kubernetes.io/projected/aae92a23-65e7-4f07-bbf8-934cc18e916e-kube-api-access-bq8t6\") pod \"aae92a23-65e7-4f07-bbf8-934cc18e916e\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") "
Oct 30 22:10:27.407406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.407396    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-data" pod "aae92a23-65e7-4f07-bbf8-934cc18e916e" (UID: "aae92a23-65e7-4f07-bbf8-934cc18e916e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:27.407573 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.407424    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-token\" (UniqueName: \"kubernetes.io/projected/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-token\") pod \"aae92a23-65e7-4f07-bbf8-934cc18e916e\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") "
Oct 30 22:10:27.407573 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.407451    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aae92a23-65e7-4f07-bbf8-934cc18e916e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aae92a23-65e7-4f07-bbf8-934cc18e916e-istiod-ca-cert\") pod \"aae92a23-65e7-4f07-bbf8-934cc18e916e\" (UID: \"aae92a23-65e7-4f07-bbf8-934cc18e916e\") "
Oct 30 22:10:27.407573 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.407558    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-workload-socket" pod "aae92a23-65e7-4f07-bbf8-934cc18e916e" (UID: "aae92a23-65e7-4f07-bbf8-934cc18e916e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:27.407712 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.407571    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-credential-socket" pod "aae92a23-65e7-4f07-bbf8-934cc18e916e" (UID: "aae92a23-65e7-4f07-bbf8-934cc18e916e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:27.407889 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.407865    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aae92a23-65e7-4f07-bbf8-934cc18e916e-istiod-ca-cert" pod "aae92a23-65e7-4f07-bbf8-934cc18e916e" (UID: "aae92a23-65e7-4f07-bbf8-934cc18e916e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:27.408128 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.408099    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aae92a23-65e7-4f07-bbf8-934cc18e916e-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.408214 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.408127    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.408214 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.408144    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.408214 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.408158    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.408214 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.408171    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.409624 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.409599    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-podinfo" pod "aae92a23-65e7-4f07-bbf8-934cc18e916e" (UID: "aae92a23-65e7-4f07-bbf8-934cc18e916e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:10:27.409717 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.409625    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aae92a23-65e7-4f07-bbf8-934cc18e916e-kube-api-access-bq8t6" pod "aae92a23-65e7-4f07-bbf8-934cc18e916e" (UID: "aae92a23-65e7-4f07-bbf8-934cc18e916e"). InnerVolumeSpecName "kube-api-access-bq8t6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:27.409717 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.409647    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-envoy" pod "aae92a23-65e7-4f07-bbf8-934cc18e916e" (UID: "aae92a23-65e7-4f07-bbf8-934cc18e916e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:27.409995 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.409979    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-token" pod "aae92a23-65e7-4f07-bbf8-934cc18e916e" (UID: "aae92a23-65e7-4f07-bbf8-934cc18e916e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:27.510062 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.510023    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.510062 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.510056    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.510062 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.510069    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bq8t6\" (UniqueName: \"kubernetes.io/projected/aae92a23-65e7-4f07-bbf8-934cc18e916e-kube-api-access-bq8t6\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.510360 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.510082    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aae92a23-65e7-4f07-bbf8-934cc18e916e-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:27.928032 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:27.927987    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:28.139170 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:28.139142    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"
Oct 30 22:10:28.139684 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:28.139184    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb" event={"ID":"aae92a23-65e7-4f07-bbf8-934cc18e916e","Type":"ContainerDied","Data":"a0c08a8859824feaa01ffb9d6885c8692e1e2620b4385933e0551daf0398e969"}
Oct 30 22:10:28.139684 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:28.139292    5586 scope.go:122] "RemoveContainer" containerID="8d103da0d07c17ada9f15c90d901c2beabf81a9d856f7d1894001544b7fc1814"
Oct 30 22:10:28.352991 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:28.352909    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"]
Oct 30 22:10:28.407429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:28.407332    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-78cf83-openshift-default-c5b4b7cb7-t9jqb"]
Oct 30 22:10:29.211403 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:29.211357    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:29.376285 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:29.376244    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:29.448830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:29.448795    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aae92a23-65e7-4f07-bbf8-934cc18e916e" path="/var/lib/kubelet/pods/aae92a23-65e7-4f07-bbf8-934cc18e916e/volumes"
Oct 30 22:10:29.557888 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:29.557789    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:31.163712 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:31.163674    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:32.445857 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:32.445822    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:32.655264 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:32.655203    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:33.618921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:33.618886    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"]
Oct 30 22:10:33.727764 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:33.727717    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"]
Oct 30 22:10:33.997358 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:33.997326    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:34.203298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:34.203224    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:36.222827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:36.222784    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:36.512594 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:36.512497    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:36.677692 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:36.677648    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:38.625486 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:38.625352    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt" podUID="61e86a1d-3e10-4207-a9a1-d7a72072684d" containerName="payload-processing" containerID="cri-o://63771e290ed92cb7553b0faa85456a09930449c85a16dee30bd9c40fb83f330e" gracePeriod=25
Oct 30 22:10:38.728215 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:38.728139    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z" podUID="59f48521-b5bd-4e36-8200-7129da431734" containerName="payload-pre-processing" containerID="cri-o://36ac064df86a2810764f02c3de1a731cd80da176d22a1701dc50cdbbbae94499" gracePeriod=25
Oct 30 22:10:38.909970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:38.909934    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"
Oct 30 22:10:38.970505 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:38.970469    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61e86a1d-3e10-4207-a9a1-d7a72072684d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61e86a1d-3e10-4207-a9a1-d7a72072684d-plugins-config-volume\") pod \"61e86a1d-3e10-4207-a9a1-d7a72072684d\" (UID: \"61e86a1d-3e10-4207-a9a1-d7a72072684d\") "
Oct 30 22:10:38.970721 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:38.970623    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61e86a1d-3e10-4207-a9a1-d7a72072684d-kube-api-access-kz4rw\" (UniqueName: \"kubernetes.io/projected/61e86a1d-3e10-4207-a9a1-d7a72072684d-kube-api-access-kz4rw\") pod \"61e86a1d-3e10-4207-a9a1-d7a72072684d\" (UID: \"61e86a1d-3e10-4207-a9a1-d7a72072684d\") "
Oct 30 22:10:38.971258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:38.970946    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61e86a1d-3e10-4207-a9a1-d7a72072684d-plugins-config-volume" pod "61e86a1d-3e10-4207-a9a1-d7a72072684d" (UID: "61e86a1d-3e10-4207-a9a1-d7a72072684d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:38.971385 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:38.971306    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61e86a1d-3e10-4207-a9a1-d7a72072684d-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:38.973095 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:38.973071    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61e86a1d-3e10-4207-a9a1-d7a72072684d-kube-api-access-kz4rw" pod "61e86a1d-3e10-4207-a9a1-d7a72072684d" (UID: "61e86a1d-3e10-4207-a9a1-d7a72072684d"). InnerVolumeSpecName "kube-api-access-kz4rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:39.023692 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.023649    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"
Oct 30 22:10:39.073252 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.073211    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz4rw\" (UniqueName: \"kubernetes.io/projected/61e86a1d-3e10-4207-a9a1-d7a72072684d-kube-api-access-kz4rw\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:39.173690 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.173654    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/59f48521-b5bd-4e36-8200-7129da431734-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f48521-b5bd-4e36-8200-7129da431734-plugins-config-volume\") pod \"59f48521-b5bd-4e36-8200-7129da431734\" (UID: \"59f48521-b5bd-4e36-8200-7129da431734\") "
Oct 30 22:10:39.173938 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.173923    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59f48521-b5bd-4e36-8200-7129da431734-kube-api-access-wmnrl\" (UniqueName: \"kubernetes.io/projected/59f48521-b5bd-4e36-8200-7129da431734-kube-api-access-wmnrl\") pod \"59f48521-b5bd-4e36-8200-7129da431734\" (UID: \"59f48521-b5bd-4e36-8200-7129da431734\") "
Oct 30 22:10:39.174217 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.174184    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59f48521-b5bd-4e36-8200-7129da431734-plugins-config-volume" pod "59f48521-b5bd-4e36-8200-7129da431734" (UID: "59f48521-b5bd-4e36-8200-7129da431734"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:39.174487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.174466    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f48521-b5bd-4e36-8200-7129da431734-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:39.176220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.176188    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59f48521-b5bd-4e36-8200-7129da431734-kube-api-access-wmnrl" pod "59f48521-b5bd-4e36-8200-7129da431734" (UID: "59f48521-b5bd-4e36-8200-7129da431734"). InnerVolumeSpecName "kube-api-access-wmnrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:39.226714 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.226678    5586 generic.go:356] "Generic (PLEG): container finished" podID="59f48521-b5bd-4e36-8200-7129da431734" containerID="36ac064df86a2810764f02c3de1a731cd80da176d22a1701dc50cdbbbae94499" exitCode=0
Oct 30 22:10:39.226946 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.226739    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"
Oct 30 22:10:39.226946 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.226775    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z" event={"ID":"59f48521-b5bd-4e36-8200-7129da431734","Type":"ContainerDied","Data":"36ac064df86a2810764f02c3de1a731cd80da176d22a1701dc50cdbbbae94499"}
Oct 30 22:10:39.226946 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.226875    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z" event={"ID":"59f48521-b5bd-4e36-8200-7129da431734","Type":"ContainerDied","Data":"465d35141bd52cefee3970eadc5fa7e3f6efd25abc34c8b0b115184cdf227ebf"}
Oct 30 22:10:39.226946 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.226912    5586 scope.go:122] "RemoveContainer" containerID="36ac064df86a2810764f02c3de1a731cd80da176d22a1701dc50cdbbbae94499"
Oct 30 22:10:39.228461 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.228436    5586 generic.go:356] "Generic (PLEG): container finished" podID="61e86a1d-3e10-4207-a9a1-d7a72072684d" containerID="63771e290ed92cb7553b0faa85456a09930449c85a16dee30bd9c40fb83f330e" exitCode=0
Oct 30 22:10:39.228585 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.228482    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"
Oct 30 22:10:39.228585 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.228520    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt" event={"ID":"61e86a1d-3e10-4207-a9a1-d7a72072684d","Type":"ContainerDied","Data":"63771e290ed92cb7553b0faa85456a09930449c85a16dee30bd9c40fb83f330e"}
Oct 30 22:10:39.228585 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.228579    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt" event={"ID":"61e86a1d-3e10-4207-a9a1-d7a72072684d","Type":"ContainerDied","Data":"255b2872b2251df35f90d62c03a2e9b02d1f16c050354639c877d38788775d96"}
Oct 30 22:10:39.239130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.239105    5586 scope.go:122] "RemoveContainer" containerID="36ac064df86a2810764f02c3de1a731cd80da176d22a1701dc50cdbbbae94499"
Oct 30 22:10:39.239578 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:10:39.239553    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36ac064df86a2810764f02c3de1a731cd80da176d22a1701dc50cdbbbae94499\": container with ID starting with 36ac064df86a2810764f02c3de1a731cd80da176d22a1701dc50cdbbbae94499 not found: ID does not exist" containerID="36ac064df86a2810764f02c3de1a731cd80da176d22a1701dc50cdbbbae94499"
Oct 30 22:10:39.239678 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.239585    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36ac064df86a2810764f02c3de1a731cd80da176d22a1701dc50cdbbbae94499"} err="failed to get container status \"36ac064df86a2810764f02c3de1a731cd80da176d22a1701dc50cdbbbae94499\": rpc error: code = NotFound desc = could not find container \"36ac064df86a2810764f02c3de1a731cd80da176d22a1701dc50cdbbbae94499\": container with ID starting with 36ac064df86a2810764f02c3de1a731cd80da176d22a1701dc50cdbbbae94499 not found: ID does not exist"
Oct 30 22:10:39.239678 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.239605    5586 scope.go:122] "RemoveContainer" containerID="63771e290ed92cb7553b0faa85456a09930449c85a16dee30bd9c40fb83f330e"
Oct 30 22:10:39.250111 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.250085    5586 scope.go:122] "RemoveContainer" containerID="63771e290ed92cb7553b0faa85456a09930449c85a16dee30bd9c40fb83f330e"
Oct 30 22:10:39.250494 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:10:39.250470    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63771e290ed92cb7553b0faa85456a09930449c85a16dee30bd9c40fb83f330e\": container with ID starting with 63771e290ed92cb7553b0faa85456a09930449c85a16dee30bd9c40fb83f330e not found: ID does not exist" containerID="63771e290ed92cb7553b0faa85456a09930449c85a16dee30bd9c40fb83f330e"
Oct 30 22:10:39.250577 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.250506    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63771e290ed92cb7553b0faa85456a09930449c85a16dee30bd9c40fb83f330e"} err="failed to get container status \"63771e290ed92cb7553b0faa85456a09930449c85a16dee30bd9c40fb83f330e\": rpc error: code = NotFound desc = could not find container \"63771e290ed92cb7553b0faa85456a09930449c85a16dee30bd9c40fb83f330e\": container with ID starting with 63771e290ed92cb7553b0faa85456a09930449c85a16dee30bd9c40fb83f330e not found: ID does not exist"
Oct 30 22:10:39.275943 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.275910    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmnrl\" (UniqueName: \"kubernetes.io/projected/59f48521-b5bd-4e36-8200-7129da431734-kube-api-access-wmnrl\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:39.390105 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.390066    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"]
Oct 30 22:10:39.396806 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.396770    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9f8e92-7ddc75dc47-g982z"]
Oct 30 22:10:39.449388 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.449309    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59f48521-b5bd-4e36-8200-7129da431734" path="/var/lib/kubelet/pods/59f48521-b5bd-4e36-8200-7129da431734/volumes"
Oct 30 22:10:39.514069 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.514032    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"]
Oct 30 22:10:39.532682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:39.532648    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9f8e92-7b988cb4d5-8x8pt"]
Oct 30 22:10:40.638316 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:40.638283    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"]
Oct 30 22:10:40.638782 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:40.638498    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd" podUID="1e584555-1a5c-429b-81b2-be625654d315" containerName="maas-api" containerID="cri-o://ce486372b28b4458dbfc4f924c4c009ff6a88e4d77dc32dc7b86060462f9c7fd" gracePeriod=30
Oct 30 22:10:40.762220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:40.762185    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:40.953427 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:40.953394    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:10:41.101062 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.101017    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e584555-1a5c-429b-81b2-be625654d315-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e584555-1a5c-429b-81b2-be625654d315-maas-api-tls\") pod \"1e584555-1a5c-429b-81b2-be625654d315\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") "
Oct 30 22:10:41.101306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.101152    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e584555-1a5c-429b-81b2-be625654d315-kube-api-access-wjrkq\" (UniqueName: \"kubernetes.io/projected/1e584555-1a5c-429b-81b2-be625654d315-kube-api-access-wjrkq\") pod \"1e584555-1a5c-429b-81b2-be625654d315\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") "
Oct 30 22:10:41.101306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.101199    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e584555-1a5c-429b-81b2-be625654d315-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e584555-1a5c-429b-81b2-be625654d315-metrics-certs\") pod \"1e584555-1a5c-429b-81b2-be625654d315\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") "
Oct 30 22:10:41.101306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.101228    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e584555-1a5c-429b-81b2-be625654d315-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e584555-1a5c-429b-81b2-be625654d315-openshift-service-ca\") pod \"1e584555-1a5c-429b-81b2-be625654d315\" (UID: \"1e584555-1a5c-429b-81b2-be625654d315\") "
Oct 30 22:10:41.101729 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.101676    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e584555-1a5c-429b-81b2-be625654d315-openshift-service-ca" pod "1e584555-1a5c-429b-81b2-be625654d315" (UID: "1e584555-1a5c-429b-81b2-be625654d315"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:41.102011 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.101981    5586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e584555-1a5c-429b-81b2-be625654d315-openshift-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:41.103905 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.103877    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e584555-1a5c-429b-81b2-be625654d315-maas-api-tls" pod "1e584555-1a5c-429b-81b2-be625654d315" (UID: "1e584555-1a5c-429b-81b2-be625654d315"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:41.104036 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.103964    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e584555-1a5c-429b-81b2-be625654d315-kube-api-access-wjrkq" pod "1e584555-1a5c-429b-81b2-be625654d315" (UID: "1e584555-1a5c-429b-81b2-be625654d315"). InnerVolumeSpecName "kube-api-access-wjrkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:41.104225 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.104210    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e584555-1a5c-429b-81b2-be625654d315-metrics-certs" pod "1e584555-1a5c-429b-81b2-be625654d315" (UID: "1e584555-1a5c-429b-81b2-be625654d315"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:41.207383 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.207260    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e584555-1a5c-429b-81b2-be625654d315-maas-api-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:41.207383 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.207306    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjrkq\" (UniqueName: \"kubernetes.io/projected/1e584555-1a5c-429b-81b2-be625654d315-kube-api-access-wjrkq\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:41.207383 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.207322    5586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e584555-1a5c-429b-81b2-be625654d315-metrics-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:41.252010 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.251964    5586 generic.go:356] "Generic (PLEG): container finished" podID="1e584555-1a5c-429b-81b2-be625654d315" containerID="ce486372b28b4458dbfc4f924c4c009ff6a88e4d77dc32dc7b86060462f9c7fd" exitCode=0
Oct 30 22:10:41.252269 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.252021    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"
Oct 30 22:10:41.252269 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.252137    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd" event={"ID":"1e584555-1a5c-429b-81b2-be625654d315","Type":"ContainerDied","Data":"ce486372b28b4458dbfc4f924c4c009ff6a88e4d77dc32dc7b86060462f9c7fd"}
Oct 30 22:10:41.252269 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.252219    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd" event={"ID":"1e584555-1a5c-429b-81b2-be625654d315","Type":"ContainerDied","Data":"eb948972db4424753d05195190e5516fc68c8c8f826c0cd6e62edc86baf91064"}
Oct 30 22:10:41.252463 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.252286    5586 scope.go:122] "RemoveContainer" containerID="ce486372b28b4458dbfc4f924c4c009ff6a88e4d77dc32dc7b86060462f9c7fd"
Oct 30 22:10:41.268194 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.268165    5586 scope.go:122] "RemoveContainer" containerID="ce486372b28b4458dbfc4f924c4c009ff6a88e4d77dc32dc7b86060462f9c7fd"
Oct 30 22:10:41.268691 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:10:41.268665    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce486372b28b4458dbfc4f924c4c009ff6a88e4d77dc32dc7b86060462f9c7fd\": container with ID starting with ce486372b28b4458dbfc4f924c4c009ff6a88e4d77dc32dc7b86060462f9c7fd not found: ID does not exist" containerID="ce486372b28b4458dbfc4f924c4c009ff6a88e4d77dc32dc7b86060462f9c7fd"
Oct 30 22:10:41.268773 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.268704    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce486372b28b4458dbfc4f924c4c009ff6a88e4d77dc32dc7b86060462f9c7fd"} err="failed to get container status \"ce486372b28b4458dbfc4f924c4c009ff6a88e4d77dc32dc7b86060462f9c7fd\": rpc error: code = NotFound desc = could not find container \"ce486372b28b4458dbfc4f924c4c009ff6a88e4d77dc32dc7b86060462f9c7fd\": container with ID starting with ce486372b28b4458dbfc4f924c4c009ff6a88e4d77dc32dc7b86060462f9c7fd not found: ID does not exist"
Oct 30 22:10:41.421728 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.421691    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"]
Oct 30 22:10:41.424833 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.424805    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6cd200-c4d788cf8-b27sd"]
Oct 30 22:10:41.448932 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.448898    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e584555-1a5c-429b-81b2-be625654d315" path="/var/lib/kubelet/pods/1e584555-1a5c-429b-81b2-be625654d315/volumes"
Oct 30 22:10:41.449356 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:41.449342    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61e86a1d-3e10-4207-a9a1-d7a72072684d" path="/var/lib/kubelet/pods/61e86a1d-3e10-4207-a9a1-d7a72072684d/volumes"
Oct 30 22:10:42.477068 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:42.477038    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:42.714286 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:42.714209    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:42.890924 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:42.890841    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:43.464070 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:43.464034    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:44.916229 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:44.916191    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:47.260520 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.260484    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"]
Oct 30 22:10:47.261008 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.260716    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2" podUID="d6ec046a-92b3-4116-a7ed-dde35559ccaa" containerName="maas-api" containerID="cri-o://dd0cb42801d7f3c555f412c60b84b295988208144d71b78885b97591c49996c4" gracePeriod=30
Oct 30 22:10:47.343794 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:10:47.343760    5586 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6ec046a_92b3_4116_a7ed_dde35559ccaa.slice/crio-dd0cb42801d7f3c555f412c60b84b295988208144d71b78885b97591c49996c4.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:10:47.394502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.394459    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:47.580085 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.580053    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:10:47.729258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.723057    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6ec046a-92b3-4116-a7ed-dde35559ccaa-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ec046a-92b3-4116-a7ed-dde35559ccaa-openshift-service-ca\") pod \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") "
Oct 30 22:10:47.729258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.723277    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-metrics-certs\") pod \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") "
Oct 30 22:10:47.729258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.723339    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6ec046a-92b3-4116-a7ed-dde35559ccaa-kube-api-access-7sn88\" (UniqueName: \"kubernetes.io/projected/d6ec046a-92b3-4116-a7ed-dde35559ccaa-kube-api-access-7sn88\") pod \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") "
Oct 30 22:10:47.729258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.723368    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-maas-api-tls\") pod \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\" (UID: \"d6ec046a-92b3-4116-a7ed-dde35559ccaa\") "
Oct 30 22:10:47.729258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.728630    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6ec046a-92b3-4116-a7ed-dde35559ccaa-openshift-service-ca" pod "d6ec046a-92b3-4116-a7ed-dde35559ccaa" (UID: "d6ec046a-92b3-4116-a7ed-dde35559ccaa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:47.773187 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.773141    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-maas-api-tls" pod "d6ec046a-92b3-4116-a7ed-dde35559ccaa" (UID: "d6ec046a-92b3-4116-a7ed-dde35559ccaa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:47.773430 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.773249    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-metrics-certs" pod "d6ec046a-92b3-4116-a7ed-dde35559ccaa" (UID: "d6ec046a-92b3-4116-a7ed-dde35559ccaa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:47.775402 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.775355    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6ec046a-92b3-4116-a7ed-dde35559ccaa-kube-api-access-7sn88" pod "d6ec046a-92b3-4116-a7ed-dde35559ccaa" (UID: "d6ec046a-92b3-4116-a7ed-dde35559ccaa"). InnerVolumeSpecName "kube-api-access-7sn88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:47.825879 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.825796    5586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-metrics-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:47.825879 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.825828    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sn88\" (UniqueName: \"kubernetes.io/projected/d6ec046a-92b3-4116-a7ed-dde35559ccaa-kube-api-access-7sn88\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:47.825879 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.825839    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ec046a-92b3-4116-a7ed-dde35559ccaa-maas-api-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:47.825879 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:47.825849    5586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ec046a-92b3-4116-a7ed-dde35559ccaa-openshift-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:48.306104 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:48.306066    5586 generic.go:356] "Generic (PLEG): container finished" podID="d6ec046a-92b3-4116-a7ed-dde35559ccaa" containerID="dd0cb42801d7f3c555f412c60b84b295988208144d71b78885b97591c49996c4" exitCode=0
Oct 30 22:10:48.306566 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:48.306115    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"
Oct 30 22:10:48.306566 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:48.306142    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2" event={"ID":"d6ec046a-92b3-4116-a7ed-dde35559ccaa","Type":"ContainerDied","Data":"dd0cb42801d7f3c555f412c60b84b295988208144d71b78885b97591c49996c4"}
Oct 30 22:10:48.306566 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:48.306200    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2" event={"ID":"d6ec046a-92b3-4116-a7ed-dde35559ccaa","Type":"ContainerDied","Data":"9e39b5dde440192fb4c2d371a9b4a00b3daf56ef166debb079a0cce9f1aad465"}
Oct 30 22:10:48.306566 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:48.306223    5586 scope.go:122] "RemoveContainer" containerID="dd0cb42801d7f3c555f412c60b84b295988208144d71b78885b97591c49996c4"
Oct 30 22:10:48.317293 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:48.317273    5586 scope.go:122] "RemoveContainer" containerID="dd0cb42801d7f3c555f412c60b84b295988208144d71b78885b97591c49996c4"
Oct 30 22:10:48.317564 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:10:48.317538    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd0cb42801d7f3c555f412c60b84b295988208144d71b78885b97591c49996c4\": container with ID starting with dd0cb42801d7f3c555f412c60b84b295988208144d71b78885b97591c49996c4 not found: ID does not exist" containerID="dd0cb42801d7f3c555f412c60b84b295988208144d71b78885b97591c49996c4"
Oct 30 22:10:48.317663 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:48.317569    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd0cb42801d7f3c555f412c60b84b295988208144d71b78885b97591c49996c4"} err="failed to get container status \"dd0cb42801d7f3c555f412c60b84b295988208144d71b78885b97591c49996c4\": rpc error: code = NotFound desc = could not find container \"dd0cb42801d7f3c555f412c60b84b295988208144d71b78885b97591c49996c4\": container with ID starting with dd0cb42801d7f3c555f412c60b84b295988208144d71b78885b97591c49996c4 not found: ID does not exist"
Oct 30 22:10:48.436121 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:10:48.436080    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:10:48.486638 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:48.486599    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"]
Oct 30 22:10:48.492155 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:48.492126    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-485d1e-76d6d5fd98-nw8s2"]
Oct 30 22:10:49.104473 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:49.104437    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:49.321098 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:49.321054    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:49.449840 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:49.449805    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6ec046a-92b3-4116-a7ed-dde35559ccaa" path="/var/lib/kubelet/pods/d6ec046a-92b3-4116-a7ed-dde35559ccaa/volumes"
Oct 30 22:10:49.482215 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:49.482179    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:50.007375 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:50.007338    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:50.175912 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:50.175879    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:50.361178 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:50.361097    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"]
Oct 30 22:10:50.361607 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:50.361337    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h" podUID="89b841c7-c736-4ffe-b31e-762908d62361" containerName="istio-proxy" containerID="cri-o://4147b489f9b23333acbd87d6e29fca226e7436395d5cfb1881daef617b635142" gracePeriod=30
Oct 30 22:10:50.373555 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:50.373520    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:51.415983 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:51.415940    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:51.599499 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:51.599464    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:10:51.599659 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:51.599519    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h" podUID="89b841c7-c736-4ffe-b31e-762908d62361" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:10:51.600511 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:51.600464    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:53.738163 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:53.738132    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"]
Oct 30 22:10:53.738615 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:53.738349    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4" podUID="603aeab9-aac6-4ed9-8336-08f7c2acdd6c" containerName="maas-api" containerID="cri-o://a5570b864e8a4ebaea90e8d28c39ecf1cef4a51132f8e5373a6434b869c5389f" gracePeriod=30
Oct 30 22:10:53.871545 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:53.871485    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:54.074732 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.074698    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:10:54.131932 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.131903    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-maas-api-tls\") pod \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") "
Oct 30 22:10:54.132131 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.131992    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-openshift-service-ca\") pod \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") "
Oct 30 22:10:54.132131 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.132069    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-kube-api-access-xdhrl\" (UniqueName: \"kubernetes.io/projected/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-kube-api-access-xdhrl\") pod \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") "
Oct 30 22:10:54.132131 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.132100    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-metrics-certs\") pod \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\" (UID: \"603aeab9-aac6-4ed9-8336-08f7c2acdd6c\") "
Oct 30 22:10:54.132431 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.132396    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-openshift-service-ca" pod "603aeab9-aac6-4ed9-8336-08f7c2acdd6c" (UID: "603aeab9-aac6-4ed9-8336-08f7c2acdd6c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:54.132704 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.132680    5586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-openshift-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:54.134284 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.134223    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-maas-api-tls" pod "603aeab9-aac6-4ed9-8336-08f7c2acdd6c" (UID: "603aeab9-aac6-4ed9-8336-08f7c2acdd6c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:54.134377 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.134265    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-metrics-certs" pod "603aeab9-aac6-4ed9-8336-08f7c2acdd6c" (UID: "603aeab9-aac6-4ed9-8336-08f7c2acdd6c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:54.134445 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.134403    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-kube-api-access-xdhrl" pod "603aeab9-aac6-4ed9-8336-08f7c2acdd6c" (UID: "603aeab9-aac6-4ed9-8336-08f7c2acdd6c"). InnerVolumeSpecName "kube-api-access-xdhrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:54.233980 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.233947    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-maas-api-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:54.233980 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.233981    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdhrl\" (UniqueName: \"kubernetes.io/projected/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-kube-api-access-xdhrl\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:54.234187 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.233995    5586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/603aeab9-aac6-4ed9-8336-08f7c2acdd6c-metrics-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:54.356339 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.356229    5586 generic.go:356] "Generic (PLEG): container finished" podID="603aeab9-aac6-4ed9-8336-08f7c2acdd6c" containerID="a5570b864e8a4ebaea90e8d28c39ecf1cef4a51132f8e5373a6434b869c5389f" exitCode=0
Oct 30 22:10:54.356339 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.356290    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"
Oct 30 22:10:54.356339 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.356306    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4" event={"ID":"603aeab9-aac6-4ed9-8336-08f7c2acdd6c","Type":"ContainerDied","Data":"a5570b864e8a4ebaea90e8d28c39ecf1cef4a51132f8e5373a6434b869c5389f"}
Oct 30 22:10:54.356611 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.356373    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4" event={"ID":"603aeab9-aac6-4ed9-8336-08f7c2acdd6c","Type":"ContainerDied","Data":"6d7dda6db40bea92f67591b954e3421a4dc3524abf0fc8dc870913d9db792279"}
Oct 30 22:10:54.356611 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.356402    5586 scope.go:122] "RemoveContainer" containerID="a5570b864e8a4ebaea90e8d28c39ecf1cef4a51132f8e5373a6434b869c5389f"
Oct 30 22:10:54.373924 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.373898    5586 scope.go:122] "RemoveContainer" containerID="a5570b864e8a4ebaea90e8d28c39ecf1cef4a51132f8e5373a6434b869c5389f"
Oct 30 22:10:54.374520 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:10:54.374405    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5570b864e8a4ebaea90e8d28c39ecf1cef4a51132f8e5373a6434b869c5389f\": container with ID starting with a5570b864e8a4ebaea90e8d28c39ecf1cef4a51132f8e5373a6434b869c5389f not found: ID does not exist" containerID="a5570b864e8a4ebaea90e8d28c39ecf1cef4a51132f8e5373a6434b869c5389f"
Oct 30 22:10:54.374520 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.374463    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5570b864e8a4ebaea90e8d28c39ecf1cef4a51132f8e5373a6434b869c5389f"} err="failed to get container status \"a5570b864e8a4ebaea90e8d28c39ecf1cef4a51132f8e5373a6434b869c5389f\": rpc error: code = NotFound desc = could not find container \"a5570b864e8a4ebaea90e8d28c39ecf1cef4a51132f8e5373a6434b869c5389f\": container with ID starting with a5570b864e8a4ebaea90e8d28c39ecf1cef4a51132f8e5373a6434b869c5389f not found: ID does not exist"
Oct 30 22:10:54.520083 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.520050    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"]
Oct 30 22:10:54.526371 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:54.526343    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-cr9d4"]
Oct 30 22:10:55.449651 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.449614    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="603aeab9-aac6-4ed9-8336-08f7c2acdd6c" path="/var/lib/kubelet/pods/603aeab9-aac6-4ed9-8336-08f7c2acdd6c/volumes"
Oct 30 22:10:55.600294 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.600219    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:55.643165 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.643141    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:10:55.756159 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.756125    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-istio-envoy\") pod \"89b841c7-c736-4ffe-b31e-762908d62361\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") "
Oct 30 22:10:55.756376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.756267    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89b841c7-c736-4ffe-b31e-762908d62361-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89b841c7-c736-4ffe-b31e-762908d62361-istiod-ca-cert\") pod \"89b841c7-c736-4ffe-b31e-762908d62361\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") "
Oct 30 22:10:55.756376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.756303    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-workload-certs\") pod \"89b841c7-c736-4ffe-b31e-762908d62361\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") "
Oct 30 22:10:55.756376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.756333    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-credential-socket\") pod \"89b841c7-c736-4ffe-b31e-762908d62361\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") "
Oct 30 22:10:55.756549 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.756383    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89b841c7-c736-4ffe-b31e-762908d62361-istio-token\" (UniqueName: \"kubernetes.io/projected/89b841c7-c736-4ffe-b31e-762908d62361-istio-token\") pod \"89b841c7-c736-4ffe-b31e-762908d62361\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") "
Oct 30 22:10:55.756549 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.756430    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89b841c7-c736-4ffe-b31e-762908d62361-kube-api-access-s2sx9\" (UniqueName: \"kubernetes.io/projected/89b841c7-c736-4ffe-b31e-762908d62361-kube-api-access-s2sx9\") pod \"89b841c7-c736-4ffe-b31e-762908d62361\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") "
Oct 30 22:10:55.756549 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.756518    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-workload-socket\") pod \"89b841c7-c736-4ffe-b31e-762908d62361\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") "
Oct 30 22:10:55.756706 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.756568    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/89b841c7-c736-4ffe-b31e-762908d62361-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89b841c7-c736-4ffe-b31e-762908d62361-istio-podinfo\") pod \"89b841c7-c736-4ffe-b31e-762908d62361\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") "
Oct 30 22:10:55.756706 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.756617    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-istio-data\") pod \"89b841c7-c736-4ffe-b31e-762908d62361\" (UID: \"89b841c7-c736-4ffe-b31e-762908d62361\") "
Oct 30 22:10:55.756706 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.756636    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-workload-certs" pod "89b841c7-c736-4ffe-b31e-762908d62361" (UID: "89b841c7-c736-4ffe-b31e-762908d62361"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:55.756706 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.756690    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-credential-socket" pod "89b841c7-c736-4ffe-b31e-762908d62361" (UID: "89b841c7-c736-4ffe-b31e-762908d62361"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:55.756909 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.756734    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89b841c7-c736-4ffe-b31e-762908d62361-istiod-ca-cert" pod "89b841c7-c736-4ffe-b31e-762908d62361" (UID: "89b841c7-c736-4ffe-b31e-762908d62361"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:55.756909 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.756898    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-workload-socket" pod "89b841c7-c736-4ffe-b31e-762908d62361" (UID: "89b841c7-c736-4ffe-b31e-762908d62361"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:55.756988 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.756924    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-istio-data" pod "89b841c7-c736-4ffe-b31e-762908d62361" (UID: "89b841c7-c736-4ffe-b31e-762908d62361"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:55.757479 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.757451    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:55.757599 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.757481    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:55.757599 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.757498    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89b841c7-c736-4ffe-b31e-762908d62361-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:55.757599 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.757513    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:55.757599 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.757528    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:55.758666 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.758625    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-istio-envoy" pod "89b841c7-c736-4ffe-b31e-762908d62361" (UID: "89b841c7-c736-4ffe-b31e-762908d62361"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:55.758774 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.758683    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89b841c7-c736-4ffe-b31e-762908d62361-kube-api-access-s2sx9" pod "89b841c7-c736-4ffe-b31e-762908d62361" (UID: "89b841c7-c736-4ffe-b31e-762908d62361"). InnerVolumeSpecName "kube-api-access-s2sx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:55.758774 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.758719    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89b841c7-c736-4ffe-b31e-762908d62361-istio-token" pod "89b841c7-c736-4ffe-b31e-762908d62361" (UID: "89b841c7-c736-4ffe-b31e-762908d62361"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:55.758887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.758873    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/89b841c7-c736-4ffe-b31e-762908d62361-istio-podinfo" pod "89b841c7-c736-4ffe-b31e-762908d62361" (UID: "89b841c7-c736-4ffe-b31e-762908d62361"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:10:55.859271 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.859189    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89b841c7-c736-4ffe-b31e-762908d62361-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:55.859271 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.859266    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89b841c7-c736-4ffe-b31e-762908d62361-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:55.859507 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.859286    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2sx9\" (UniqueName: \"kubernetes.io/projected/89b841c7-c736-4ffe-b31e-762908d62361-kube-api-access-s2sx9\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:55.859507 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.859302    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89b841c7-c736-4ffe-b31e-762908d62361-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:55.872213 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:55.872097    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:56.027756 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:56.027335    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:56.374739 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:56.374697    5586 generic.go:356] "Generic (PLEG): container finished" podID="89b841c7-c736-4ffe-b31e-762908d62361" containerID="4147b489f9b23333acbd87d6e29fca226e7436395d5cfb1881daef617b635142" exitCode=0
Oct 30 22:10:56.374739 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:56.374738    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h" event={"ID":"89b841c7-c736-4ffe-b31e-762908d62361","Type":"ContainerDied","Data":"4147b489f9b23333acbd87d6e29fca226e7436395d5cfb1881daef617b635142"}
Oct 30 22:10:56.375071 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:56.374754    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"
Oct 30 22:10:56.375071 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:56.374788    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h" event={"ID":"89b841c7-c736-4ffe-b31e-762908d62361","Type":"ContainerDied","Data":"dc4ed49844729e51aca3063cd9bd0e250555bcb1c951204b9f0abd4d612eb1d4"}
Oct 30 22:10:56.375071 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:56.374818    5586 scope.go:122] "RemoveContainer" containerID="4147b489f9b23333acbd87d6e29fca226e7436395d5cfb1881daef617b635142"
Oct 30 22:10:56.387357 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:56.387335    5586 scope.go:122] "RemoveContainer" containerID="a410127d6bfb122a0f4875cd14b5d6737ebce0c77e381e0d91f7e61b551bca62"
Oct 30 22:10:56.397994 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:56.397972    5586 scope.go:122] "RemoveContainer" containerID="4147b489f9b23333acbd87d6e29fca226e7436395d5cfb1881daef617b635142"
Oct 30 22:10:56.398363 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:10:56.398341    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4147b489f9b23333acbd87d6e29fca226e7436395d5cfb1881daef617b635142\": container with ID starting with 4147b489f9b23333acbd87d6e29fca226e7436395d5cfb1881daef617b635142 not found: ID does not exist" containerID="4147b489f9b23333acbd87d6e29fca226e7436395d5cfb1881daef617b635142"
Oct 30 22:10:56.398426 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:56.398373    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4147b489f9b23333acbd87d6e29fca226e7436395d5cfb1881daef617b635142"} err="failed to get container status \"4147b489f9b23333acbd87d6e29fca226e7436395d5cfb1881daef617b635142\": rpc error: code = NotFound desc = could not find container \"4147b489f9b23333acbd87d6e29fca226e7436395d5cfb1881daef617b635142\": container with ID starting with 4147b489f9b23333acbd87d6e29fca226e7436395d5cfb1881daef617b635142 not found: ID does not exist"
Oct 30 22:10:56.398426 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:56.398392    5586 scope.go:122] "RemoveContainer" containerID="a410127d6bfb122a0f4875cd14b5d6737ebce0c77e381e0d91f7e61b551bca62"
Oct 30 22:10:56.398668 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:10:56.398643    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a410127d6bfb122a0f4875cd14b5d6737ebce0c77e381e0d91f7e61b551bca62\": container with ID starting with a410127d6bfb122a0f4875cd14b5d6737ebce0c77e381e0d91f7e61b551bca62 not found: ID does not exist" containerID="a410127d6bfb122a0f4875cd14b5d6737ebce0c77e381e0d91f7e61b551bca62"
Oct 30 22:10:56.398725 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:56.398684    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a410127d6bfb122a0f4875cd14b5d6737ebce0c77e381e0d91f7e61b551bca62"} err="failed to get container status \"a410127d6bfb122a0f4875cd14b5d6737ebce0c77e381e0d91f7e61b551bca62\": rpc error: code = NotFound desc = could not find container \"a410127d6bfb122a0f4875cd14b5d6737ebce0c77e381e0d91f7e61b551bca62\": container with ID starting with a410127d6bfb122a0f4875cd14b5d6737ebce0c77e381e0d91f7e61b551bca62 not found: ID does not exist"
Oct 30 22:10:56.591967 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:56.591933    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"]
Oct 30 22:10:56.641528 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:56.641452    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-6cd200-openshift-default-9d764d7d7-hf24h"]
Oct 30 22:10:56.756696 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:56.756651    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:57.085914 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:57.085868    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:57.092771 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:57.092736    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"]
Oct 30 22:10:57.093094 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:57.093066    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz" podUID="6a144510-a70d-4553-8717-b048d64094b3" containerName="istio-proxy" containerID="cri-o://b3eef186160acd26444b392b949e5fa63e0d3d47a15c01b80d9abae3181fa070" gracePeriod=30
Oct 30 22:10:57.251285 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:57.250821    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:10:57.447869 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:57.447827    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89b841c7-c736-4ffe-b31e-762908d62361" path="/var/lib/kubelet/pods/89b841c7-c736-4ffe-b31e-762908d62361/volumes"
Oct 30 22:10:59.493660 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:59.493621    5586 patch_prober.go:29] interesting pod/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:10:59.494064 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:10:59.493676    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz" podUID="6a144510-a70d-4553-8717-b048d64094b3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:11:01.171380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:01.171346    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:02.369073 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.369009    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:11:02.421227 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.421194    5586 generic.go:356] "Generic (PLEG): container finished" podID="6a144510-a70d-4553-8717-b048d64094b3" containerID="b3eef186160acd26444b392b949e5fa63e0d3d47a15c01b80d9abae3181fa070" exitCode=0
Oct 30 22:11:02.421408 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.421271    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz" event={"ID":"6a144510-a70d-4553-8717-b048d64094b3","Type":"ContainerDied","Data":"b3eef186160acd26444b392b949e5fa63e0d3d47a15c01b80d9abae3181fa070"}
Oct 30 22:11:02.421408 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.421286    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"
Oct 30 22:11:02.421408 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.421336    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz" event={"ID":"6a144510-a70d-4553-8717-b048d64094b3","Type":"ContainerDied","Data":"2ba8007ba53378b9c0ef2d1407eb70d6092543eb22eb758c10fb713200b7d58d"}
Oct 30 22:11:02.421408 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.421371    5586 scope.go:122] "RemoveContainer" containerID="b3eef186160acd26444b392b949e5fa63e0d3d47a15c01b80d9abae3181fa070"
Oct 30 22:11:02.431762 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.431742    5586 scope.go:122] "RemoveContainer" containerID="b842587bdde38aae6135e1d4d5757ec25c2c859789ed08dbf1b215d1390f49d3"
Oct 30 22:11:02.441351 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.441329    5586 scope.go:122] "RemoveContainer" containerID="b3eef186160acd26444b392b949e5fa63e0d3d47a15c01b80d9abae3181fa070"
Oct 30 22:11:02.441626 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:11:02.441602    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3eef186160acd26444b392b949e5fa63e0d3d47a15c01b80d9abae3181fa070\": container with ID starting with b3eef186160acd26444b392b949e5fa63e0d3d47a15c01b80d9abae3181fa070 not found: ID does not exist" containerID="b3eef186160acd26444b392b949e5fa63e0d3d47a15c01b80d9abae3181fa070"
Oct 30 22:11:02.441680 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.441635    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3eef186160acd26444b392b949e5fa63e0d3d47a15c01b80d9abae3181fa070"} err="failed to get container status \"b3eef186160acd26444b392b949e5fa63e0d3d47a15c01b80d9abae3181fa070\": rpc error: code = NotFound desc = could not find container \"b3eef186160acd26444b392b949e5fa63e0d3d47a15c01b80d9abae3181fa070\": container with ID starting with b3eef186160acd26444b392b949e5fa63e0d3d47a15c01b80d9abae3181fa070 not found: ID does not exist"
Oct 30 22:11:02.441680 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.441653    5586 scope.go:122] "RemoveContainer" containerID="b842587bdde38aae6135e1d4d5757ec25c2c859789ed08dbf1b215d1390f49d3"
Oct 30 22:11:02.441912 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:11:02.441893    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b842587bdde38aae6135e1d4d5757ec25c2c859789ed08dbf1b215d1390f49d3\": container with ID starting with b842587bdde38aae6135e1d4d5757ec25c2c859789ed08dbf1b215d1390f49d3 not found: ID does not exist" containerID="b842587bdde38aae6135e1d4d5757ec25c2c859789ed08dbf1b215d1390f49d3"
Oct 30 22:11:02.441961 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.441917    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b842587bdde38aae6135e1d4d5757ec25c2c859789ed08dbf1b215d1390f49d3"} err="failed to get container status \"b842587bdde38aae6135e1d4d5757ec25c2c859789ed08dbf1b215d1390f49d3\": rpc error: code = NotFound desc = could not find container \"b842587bdde38aae6135e1d4d5757ec25c2c859789ed08dbf1b215d1390f49d3\": container with ID starting with b842587bdde38aae6135e1d4d5757ec25c2c859789ed08dbf1b215d1390f49d3 not found: ID does not exist"
Oct 30 22:11:02.449371 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.449346    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a144510-a70d-4553-8717-b048d64094b3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a144510-a70d-4553-8717-b048d64094b3-istiod-ca-cert\") pod \"6a144510-a70d-4553-8717-b048d64094b3\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") "
Oct 30 22:11:02.449523 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.449421    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6a144510-a70d-4553-8717-b048d64094b3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a144510-a70d-4553-8717-b048d64094b3-istio-podinfo\") pod \"6a144510-a70d-4553-8717-b048d64094b3\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") "
Oct 30 22:11:02.449523 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.449468    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-workload-socket\") pod \"6a144510-a70d-4553-8717-b048d64094b3\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") "
Oct 30 22:11:02.449628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.449532    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-istio-data\") pod \"6a144510-a70d-4553-8717-b048d64094b3\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") "
Oct 30 22:11:02.449628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.449609    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-istio-envoy\") pod \"6a144510-a70d-4553-8717-b048d64094b3\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") "
Oct 30 22:11:02.449744 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.449668    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-credential-socket\") pod \"6a144510-a70d-4553-8717-b048d64094b3\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") "
Oct 30 22:11:02.449744 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.449720    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a144510-a70d-4553-8717-b048d64094b3-kube-api-access-x8cwx\" (UniqueName: \"kubernetes.io/projected/6a144510-a70d-4553-8717-b048d64094b3-kube-api-access-x8cwx\") pod \"6a144510-a70d-4553-8717-b048d64094b3\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") "
Oct 30 22:11:02.449744 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.449726    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a144510-a70d-4553-8717-b048d64094b3-istiod-ca-cert" pod "6a144510-a70d-4553-8717-b048d64094b3" (UID: "6a144510-a70d-4553-8717-b048d64094b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:02.450203 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.449830    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-workload-socket" pod "6a144510-a70d-4553-8717-b048d64094b3" (UID: "6a144510-a70d-4553-8717-b048d64094b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:02.450312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.449864    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-istio-data" pod "6a144510-a70d-4553-8717-b048d64094b3" (UID: "6a144510-a70d-4553-8717-b048d64094b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:02.450312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.449756    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-workload-certs\") pod \"6a144510-a70d-4553-8717-b048d64094b3\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") "
Oct 30 22:11:02.450312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.449991    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-workload-certs" pod "6a144510-a70d-4553-8717-b048d64094b3" (UID: "6a144510-a70d-4553-8717-b048d64094b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:02.450312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.450066    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-credential-socket" pod "6a144510-a70d-4553-8717-b048d64094b3" (UID: "6a144510-a70d-4553-8717-b048d64094b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:02.450312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.450295    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a144510-a70d-4553-8717-b048d64094b3-istio-token\" (UniqueName: \"kubernetes.io/projected/6a144510-a70d-4553-8717-b048d64094b3-istio-token\") pod \"6a144510-a70d-4553-8717-b048d64094b3\" (UID: \"6a144510-a70d-4553-8717-b048d64094b3\") "
Oct 30 22:11:02.450823 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.450799    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a144510-a70d-4553-8717-b048d64094b3-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:02.450823 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.450829    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:02.450988 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.450847    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:02.450988 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.450862    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:02.450988 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.450877    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:02.452122 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.452025    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a144510-a70d-4553-8717-b048d64094b3-kube-api-access-x8cwx" pod "6a144510-a70d-4553-8717-b048d64094b3" (UID: "6a144510-a70d-4553-8717-b048d64094b3"). InnerVolumeSpecName "kube-api-access-x8cwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:02.452122 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.452100    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a144510-a70d-4553-8717-b048d64094b3-istio-podinfo" pod "6a144510-a70d-4553-8717-b048d64094b3" (UID: "6a144510-a70d-4553-8717-b048d64094b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:02.452283 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.452138    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-istio-envoy" pod "6a144510-a70d-4553-8717-b048d64094b3" (UID: "6a144510-a70d-4553-8717-b048d64094b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:02.452459 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.452443    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a144510-a70d-4553-8717-b048d64094b3-istio-token" pod "6a144510-a70d-4553-8717-b048d64094b3" (UID: "6a144510-a70d-4553-8717-b048d64094b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:02.551717 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.551677    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8cwx\" (UniqueName: \"kubernetes.io/projected/6a144510-a70d-4553-8717-b048d64094b3-kube-api-access-x8cwx\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:02.551717 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.551709    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a144510-a70d-4553-8717-b048d64094b3-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:02.551717 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.551719    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a144510-a70d-4553-8717-b048d64094b3-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:02.552029 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.551731    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a144510-a70d-4553-8717-b048d64094b3-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:02.857221 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.857176    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"]
Oct 30 22:11:02.867624 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.867590    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-485d1e-openshift-default-7559ff8579-j45sz"]
Oct 30 22:11:02.943866 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:02.943828    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:03.120081 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:03.119991    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:03.448700 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:03.448664    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a144510-a70d-4553-8717-b048d64094b3" path="/var/lib/kubelet/pods/6a144510-a70d-4553-8717-b048d64094b3/volumes"
Oct 30 22:11:05.328047 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:05.328011    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:05.540213 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:05.540177    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:05.822100 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:05.822065    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:06.521789 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:06.521746    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:06.696093 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:06.696053    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:07.552297 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:07.552257    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:07.821714 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:07.821618    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:07.983772 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:07.983737    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:08.566669 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:08.566620    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:10.320541 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:10.320508    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:12.837149 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:12.837107    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"]
Oct 30 22:11:12.941975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:12.941929    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:13.087723 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:13.087638    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:14.525210 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:14.525169    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:14.726875 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:14.726837    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:14.921852 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:14.921789    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:16.755014 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:16.754979    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:17.880731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:17.880688    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:18.363356 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:11:18.363314    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:11:19.252904 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:19.252864    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:19.459035 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:19.459003    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:21.002737 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:21.002695    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:21.704681 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:21.704636    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"]
Oct 30 22:11:21.744300 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:21.744197    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"]
Oct 30 22:11:21.767401 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:21.767357    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal" podUID="6ea3e08f52bca920602c76ad35bcaf69" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Oct 30 22:11:21.885250 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:21.885187    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:21.916162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:21.916124    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"]
Oct 30 22:11:21.916401 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:21.916374    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2" podUID="d67ce59f-5a5b-449e-9fcb-1207b9e151d3" containerName="istio-proxy" containerID="cri-o://dff88d61eb6c4e318bd514ec21b73885922aba1339f472dcaa8cc98dfa74969c" gracePeriod=30
Oct 30 22:11:22.785997 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:22.785961    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:24.241647 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:24.241595    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:24.399194 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:24.399152    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:26.681310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:26.681244    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:26.907064 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:26.907025    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:27.260228 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.260202    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 22:11:27.397659 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.397618    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-envoy\") pod \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") "
Oct 30 22:11:27.397875 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.397728    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-workload-socket\") pod \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") "
Oct 30 22:11:27.397875 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.397779    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-token\" (UniqueName: \"kubernetes.io/projected/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-token\") pod \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") "
Oct 30 22:11:27.397875 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.397839    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-credential-socket\") pod \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") "
Oct 30 22:11:27.398061 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.397879    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-kube-api-access-f6hct\" (UniqueName: \"kubernetes.io/projected/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-kube-api-access-f6hct\") pod \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") "
Oct 30 22:11:27.398061 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.397981    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-data\") pod \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") "
Oct 30 22:11:27.398061 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.397999    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-workload-socket" pod "d67ce59f-5a5b-449e-9fcb-1207b9e151d3" (UID: "d67ce59f-5a5b-449e-9fcb-1207b9e151d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:27.398337 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.398084    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-podinfo\") pod \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") "
Oct 30 22:11:27.398337 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.398169    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istiod-ca-cert\") pod \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") "
Oct 30 22:11:27.398337 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.398178    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-credential-socket" pod "d67ce59f-5a5b-449e-9fcb-1207b9e151d3" (UID: "d67ce59f-5a5b-449e-9fcb-1207b9e151d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:27.398337 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.398209    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-workload-certs\") pod \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\" (UID: \"d67ce59f-5a5b-449e-9fcb-1207b9e151d3\") "
Oct 30 22:11:27.398337 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.398310    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-data" pod "d67ce59f-5a5b-449e-9fcb-1207b9e151d3" (UID: "d67ce59f-5a5b-449e-9fcb-1207b9e151d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:27.398666 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.398625    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-workload-certs" pod "d67ce59f-5a5b-449e-9fcb-1207b9e151d3" (UID: "d67ce59f-5a5b-449e-9fcb-1207b9e151d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:27.398757 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.398671    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istiod-ca-cert" pod "d67ce59f-5a5b-449e-9fcb-1207b9e151d3" (UID: "d67ce59f-5a5b-449e-9fcb-1207b9e151d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:27.398817 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.398794    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:27.398817 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.398811    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:27.398929 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.398826    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:27.398929 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.398834    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:27.398929 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.398843    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:27.401317 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.401286    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-kube-api-access-f6hct" pod "d67ce59f-5a5b-449e-9fcb-1207b9e151d3" (UID: "d67ce59f-5a5b-449e-9fcb-1207b9e151d3"). InnerVolumeSpecName "kube-api-access-f6hct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:27.401433 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.401337    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-token" pod "d67ce59f-5a5b-449e-9fcb-1207b9e151d3" (UID: "d67ce59f-5a5b-449e-9fcb-1207b9e151d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:27.402675 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.402642    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-envoy" pod "d67ce59f-5a5b-449e-9fcb-1207b9e151d3" (UID: "d67ce59f-5a5b-449e-9fcb-1207b9e151d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:27.405400 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.405374    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-podinfo" pod "d67ce59f-5a5b-449e-9fcb-1207b9e151d3" (UID: "d67ce59f-5a5b-449e-9fcb-1207b9e151d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:27.499676 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.499581    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:27.499676 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.499615    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:27.499676 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.499631    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:27.499676 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.499641    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6hct\" (UniqueName: \"kubernetes.io/projected/d67ce59f-5a5b-449e-9fcb-1207b9e151d3-kube-api-access-f6hct\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:27.615030 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.614990    5586 generic.go:356] "Generic (PLEG): container finished" podID="d67ce59f-5a5b-449e-9fcb-1207b9e151d3" containerID="dff88d61eb6c4e318bd514ec21b73885922aba1339f472dcaa8cc98dfa74969c" exitCode=0
Oct 30 22:11:27.615209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.615042    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"
Oct 30 22:11:27.615209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.615048    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2" event={"ID":"d67ce59f-5a5b-449e-9fcb-1207b9e151d3","Type":"ContainerDied","Data":"dff88d61eb6c4e318bd514ec21b73885922aba1339f472dcaa8cc98dfa74969c"}
Oct 30 22:11:27.615209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.615107    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2" event={"ID":"d67ce59f-5a5b-449e-9fcb-1207b9e151d3","Type":"ContainerDied","Data":"948964a6ce0e17c1941c4737821ea5d633c74e68fda2f5716c731dcf1c151a9b"}
Oct 30 22:11:27.615209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.615136    5586 scope.go:122] "RemoveContainer" containerID="dff88d61eb6c4e318bd514ec21b73885922aba1339f472dcaa8cc98dfa74969c"
Oct 30 22:11:27.627429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.627404    5586 scope.go:122] "RemoveContainer" containerID="dff88d61eb6c4e318bd514ec21b73885922aba1339f472dcaa8cc98dfa74969c"
Oct 30 22:11:27.627772 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:11:27.627750    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dff88d61eb6c4e318bd514ec21b73885922aba1339f472dcaa8cc98dfa74969c\": container with ID starting with dff88d61eb6c4e318bd514ec21b73885922aba1339f472dcaa8cc98dfa74969c not found: ID does not exist" containerID="dff88d61eb6c4e318bd514ec21b73885922aba1339f472dcaa8cc98dfa74969c"
Oct 30 22:11:27.627842 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.627789    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dff88d61eb6c4e318bd514ec21b73885922aba1339f472dcaa8cc98dfa74969c"} err="failed to get container status \"dff88d61eb6c4e318bd514ec21b73885922aba1339f472dcaa8cc98dfa74969c\": rpc error: code = NotFound desc = could not find container \"dff88d61eb6c4e318bd514ec21b73885922aba1339f472dcaa8cc98dfa74969c\": container with ID starting with dff88d61eb6c4e318bd514ec21b73885922aba1339f472dcaa8cc98dfa74969c not found: ID does not exist"
Oct 30 22:11:27.797508 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.797420    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"]
Oct 30 22:11:27.808185 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.808153    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-tnch2"]
Oct 30 22:11:27.876245 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:27.876186    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" containerID="cri-o://c674553328b2747bd7248c0f6085596197d31aa0a5bd6d52999b137b0e3114ea" gracePeriod=45
Oct 30 22:11:28.006625 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.006591    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": dial tcp 10.134.0.62:8000: connect: connection refused"
Oct 30 22:11:28.158997 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.158971    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 22:11:28.212563 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.212510    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:28.309456 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.309349    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-dshm\") pod \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") "
Oct 30 22:11:28.309623 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.309460    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-model-cache\") pod \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") "
Oct 30 22:11:28.309623 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.309563    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-tls-certs\" (UniqueName: \"kubernetes.io/secret/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-tls-certs\") pod \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") "
Oct 30 22:11:28.309757 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.309639    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-kube-api-access-8wmzk\" (UniqueName: \"kubernetes.io/projected/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-kube-api-access-8wmzk\") pod \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") "
Oct 30 22:11:28.309757 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.309679    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-tmp-dir\") pod \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") "
Oct 30 22:11:28.309757 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.309710    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-home\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-home\") pod \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\" (UID: \"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb\") "
Oct 30 22:11:28.309921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.309760    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-model-cache" pod "0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" (UID: "0546e6b9-2621-4176-a47f-1bdd9e4c4ceb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:28.310051 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.309990    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-tmp-dir" pod "0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" (UID: "0546e6b9-2621-4176-a47f-1bdd9e4c4ceb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:28.310172 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.310151    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-home" pod "0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" (UID: "0546e6b9-2621-4176-a47f-1bdd9e4c4ceb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:28.310342 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.310311    5586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-tmp-dir\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:28.310342 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.310331    5586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-home\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:28.310342 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.310345    5586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-model-cache\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:28.313276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.312424    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-tls-certs" pod "0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" (UID: "0546e6b9-2621-4176-a47f-1bdd9e4c4ceb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:11:28.313276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.312430    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-dshm" pod "0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" (UID: "0546e6b9-2621-4176-a47f-1bdd9e4c4ceb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:28.313276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.312468    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-kube-api-access-8wmzk" pod "0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" (UID: "0546e6b9-2621-4176-a47f-1bdd9e4c4ceb"). InnerVolumeSpecName "kube-api-access-8wmzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:28.362268 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.362187    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:28.411712 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.411675    5586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-dshm\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:28.411712 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.411707    5586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-tls-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:28.411712 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.411719    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wmzk\" (UniqueName: \"kubernetes.io/projected/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb-kube-api-access-8wmzk\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:28.511379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.511341    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:28.624140 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.624042    5586 generic.go:356] "Generic (PLEG): container finished" podID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerID="c674553328b2747bd7248c0f6085596197d31aa0a5bd6d52999b137b0e3114ea" exitCode=0
Oct 30 22:11:28.624140 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.624092    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"
Oct 30 22:11:28.624140 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.624132    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" event={"ID":"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb","Type":"ContainerDied","Data":"c674553328b2747bd7248c0f6085596197d31aa0a5bd6d52999b137b0e3114ea"}
Oct 30 22:11:28.624445 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.624175    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp" event={"ID":"0546e6b9-2621-4176-a47f-1bdd9e4c4ceb","Type":"ContainerDied","Data":"9f91bd1514c3c6f1ce8544572b6da4e8409d3f0eda40bbb6f11e593a822f342a"}
Oct 30 22:11:28.624445 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.624194    5586 scope.go:122] "RemoveContainer" containerID="c674553328b2747bd7248c0f6085596197d31aa0a5bd6d52999b137b0e3114ea"
Oct 30 22:11:28.635020 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.634997    5586 scope.go:122] "RemoveContainer" containerID="c674553328b2747bd7248c0f6085596197d31aa0a5bd6d52999b137b0e3114ea"
Oct 30 22:11:28.635308 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:11:28.635287    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c674553328b2747bd7248c0f6085596197d31aa0a5bd6d52999b137b0e3114ea\": container with ID starting with c674553328b2747bd7248c0f6085596197d31aa0a5bd6d52999b137b0e3114ea not found: ID does not exist" containerID="c674553328b2747bd7248c0f6085596197d31aa0a5bd6d52999b137b0e3114ea"
Oct 30 22:11:28.635374 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.635314    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c674553328b2747bd7248c0f6085596197d31aa0a5bd6d52999b137b0e3114ea"} err="failed to get container status \"c674553328b2747bd7248c0f6085596197d31aa0a5bd6d52999b137b0e3114ea\": rpc error: code = NotFound desc = could not find container \"c674553328b2747bd7248c0f6085596197d31aa0a5bd6d52999b137b0e3114ea\": container with ID starting with c674553328b2747bd7248c0f6085596197d31aa0a5bd6d52999b137b0e3114ea not found: ID does not exist"
Oct 30 22:11:28.790268 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.790213    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"]
Oct 30 22:11:28.833032 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:28.833000    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-dda9f5/premium-w0-dda9f5-kserve-6b4f99d667-7fnpp"]
Oct 30 22:11:29.457500 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:29.457461    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" path="/var/lib/kubelet/pods/0546e6b9-2621-4176-a47f-1bdd9e4c4ceb/volumes"
Oct 30 22:11:29.457964 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:29.457950    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d67ce59f-5a5b-449e-9fcb-1207b9e151d3" path="/var/lib/kubelet/pods/d67ce59f-5a5b-449e-9fcb-1207b9e151d3/volumes"
Oct 30 22:11:32.679670 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:32.679624    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:34.271314 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:34.271262    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:34.424646 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:34.424610    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:34.613623 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:34.613539    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:35.205955 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:35.205925    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:36.750318 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:36.750270    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main" containerID="cri-o://e23d8d6f66078da4de7aa5c6c716a7bcaf032fea1aec5caa2809de3b91b163b7" gracePeriod=45
Oct 30 22:11:36.765834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:36.765796    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" containerID="cri-o://affa215878658387953b994491a29506ff0ada3fe72211fe8501492bf60d4838" gracePeriod=45
Oct 30 22:11:37.057886 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.057861    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 22:11:37.061069 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.061048    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 22:11:37.119490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.119463    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-tmp-dir\") pod \"2eebc209-0c68-4e4c-9485-466eb87d6154\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") "
Oct 30 22:11:37.119685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.119499    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-tmp-dir\") pod \"6e412c89-24a0-474e-9e20-6be984b88f3e\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") "
Oct 30 22:11:37.119685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.119565    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-home\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-home\") pod \"6e412c89-24a0-474e-9e20-6be984b88f3e\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") "
Oct 30 22:11:37.119685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.119627    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-model-cache\") pod \"6e412c89-24a0-474e-9e20-6be984b88f3e\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") "
Oct 30 22:11:37.119850 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.119696    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-dshm\") pod \"2eebc209-0c68-4e4c-9485-466eb87d6154\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") "
Oct 30 22:11:37.119850 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.119738    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-tmp-dir" pod "2eebc209-0c68-4e4c-9485-466eb87d6154" (UID: "2eebc209-0c68-4e4c-9485-466eb87d6154"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:37.119850 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.119765    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6e412c89-24a0-474e-9e20-6be984b88f3e-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e412c89-24a0-474e-9e20-6be984b88f3e-tls-certs\") pod \"6e412c89-24a0-474e-9e20-6be984b88f3e\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") "
Oct 30 22:11:37.119850 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.119828    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2eebc209-0c68-4e4c-9485-466eb87d6154-kube-api-access-q9jwh\" (UniqueName: \"kubernetes.io/projected/2eebc209-0c68-4e4c-9485-466eb87d6154-kube-api-access-q9jwh\") pod \"2eebc209-0c68-4e4c-9485-466eb87d6154\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") "
Oct 30 22:11:37.120054 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.119823    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-tmp-dir" pod "6e412c89-24a0-474e-9e20-6be984b88f3e" (UID: "6e412c89-24a0-474e-9e20-6be984b88f3e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:37.120054 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.119867    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-dshm\") pod \"6e412c89-24a0-474e-9e20-6be984b88f3e\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") "
Oct 30 22:11:37.120054 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.119896    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-model-cache" pod "6e412c89-24a0-474e-9e20-6be984b88f3e" (UID: "6e412c89-24a0-474e-9e20-6be984b88f3e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:37.120054 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.119912    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-home" pod "6e412c89-24a0-474e-9e20-6be984b88f3e" (UID: "6e412c89-24a0-474e-9e20-6be984b88f3e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:37.120054 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.119939    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-home\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-home\") pod \"2eebc209-0c68-4e4c-9485-466eb87d6154\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") "
Oct 30 22:11:37.120054 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.120011    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-model-cache\") pod \"2eebc209-0c68-4e4c-9485-466eb87d6154\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") "
Oct 30 22:11:37.120374 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.120067    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e412c89-24a0-474e-9e20-6be984b88f3e-kube-api-access-5g4kr\" (UniqueName: \"kubernetes.io/projected/6e412c89-24a0-474e-9e20-6be984b88f3e-kube-api-access-5g4kr\") pod \"6e412c89-24a0-474e-9e20-6be984b88f3e\" (UID: \"6e412c89-24a0-474e-9e20-6be984b88f3e\") "
Oct 30 22:11:37.120374 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.120099    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2eebc209-0c68-4e4c-9485-466eb87d6154-tls-certs\" (UniqueName: \"kubernetes.io/secret/2eebc209-0c68-4e4c-9485-466eb87d6154-tls-certs\") pod \"2eebc209-0c68-4e4c-9485-466eb87d6154\" (UID: \"2eebc209-0c68-4e4c-9485-466eb87d6154\") "
Oct 30 22:11:37.120374 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.120207    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-home" pod "2eebc209-0c68-4e4c-9485-466eb87d6154" (UID: "2eebc209-0c68-4e4c-9485-466eb87d6154"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:37.120816 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.120714    5586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-home\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.120816 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.120744    5586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-model-cache\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.120816 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.120762    5586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-home\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.120816 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.120776    5586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-tmp-dir\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.120816 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.120791    5586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-tmp-dir\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.121174 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.120994    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-model-cache" pod "2eebc209-0c68-4e4c-9485-466eb87d6154" (UID: "2eebc209-0c68-4e4c-9485-466eb87d6154"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:37.122359 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.122323    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eebc209-0c68-4e4c-9485-466eb87d6154-kube-api-access-q9jwh" pod "2eebc209-0c68-4e4c-9485-466eb87d6154" (UID: "2eebc209-0c68-4e4c-9485-466eb87d6154"). InnerVolumeSpecName "kube-api-access-q9jwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:37.122492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.122451    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-dshm" pod "6e412c89-24a0-474e-9e20-6be984b88f3e" (UID: "6e412c89-24a0-474e-9e20-6be984b88f3e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:37.122492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.122459    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e412c89-24a0-474e-9e20-6be984b88f3e-tls-certs" pod "6e412c89-24a0-474e-9e20-6be984b88f3e" (UID: "6e412c89-24a0-474e-9e20-6be984b88f3e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:11:37.122816 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.122793    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e412c89-24a0-474e-9e20-6be984b88f3e-kube-api-access-5g4kr" pod "6e412c89-24a0-474e-9e20-6be984b88f3e" (UID: "6e412c89-24a0-474e-9e20-6be984b88f3e"). InnerVolumeSpecName "kube-api-access-5g4kr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:37.122902 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.122871    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-dshm" pod "2eebc209-0c68-4e4c-9485-466eb87d6154" (UID: "2eebc209-0c68-4e4c-9485-466eb87d6154"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:37.123029 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.123012    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2eebc209-0c68-4e4c-9485-466eb87d6154-tls-certs" pod "2eebc209-0c68-4e4c-9485-466eb87d6154" (UID: "2eebc209-0c68-4e4c-9485-466eb87d6154"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:11:37.222324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.222290    5586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-model-cache\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.222324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.222325    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g4kr\" (UniqueName: \"kubernetes.io/projected/6e412c89-24a0-474e-9e20-6be984b88f3e-kube-api-access-5g4kr\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.222592 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.222338    5586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eebc209-0c68-4e4c-9485-466eb87d6154-tls-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.222592 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.222347    5586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eebc209-0c68-4e4c-9485-466eb87d6154-dshm\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.222592 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.222355    5586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e412c89-24a0-474e-9e20-6be984b88f3e-tls-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.222592 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.222363    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9jwh\" (UniqueName: \"kubernetes.io/projected/2eebc209-0c68-4e4c-9485-466eb87d6154-kube-api-access-q9jwh\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.222592 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.222371    5586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e412c89-24a0-474e-9e20-6be984b88f3e-dshm\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:37.693379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.693349    5586 generic.go:356] "Generic (PLEG): container finished" podID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerID="affa215878658387953b994491a29506ff0ada3fe72211fe8501492bf60d4838" exitCode=0
Oct 30 22:11:37.693549 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.693395    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"
Oct 30 22:11:37.693549 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.693441    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" event={"ID":"6e412c89-24a0-474e-9e20-6be984b88f3e","Type":"ContainerDied","Data":"affa215878658387953b994491a29506ff0ada3fe72211fe8501492bf60d4838"}
Oct 30 22:11:37.693549 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.693487    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" event={"ID":"6e412c89-24a0-474e-9e20-6be984b88f3e","Type":"ContainerDied","Data":"d9940845a1487f6f4447cb638ca9cba2c0dda7029a4b96cf5f44b854565fd523"}
Oct 30 22:11:37.693549 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.693508    5586 scope.go:122] "RemoveContainer" containerID="affa215878658387953b994491a29506ff0ada3fe72211fe8501492bf60d4838"
Oct 30 22:11:37.695001 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.694977    5586 generic.go:356] "Generic (PLEG): container finished" podID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerID="e23d8d6f66078da4de7aa5c6c716a7bcaf032fea1aec5caa2809de3b91b163b7" exitCode=0
Oct 30 22:11:37.695143 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.695036    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"
Oct 30 22:11:37.695143 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.695085    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" event={"ID":"2eebc209-0c68-4e4c-9485-466eb87d6154","Type":"ContainerDied","Data":"e23d8d6f66078da4de7aa5c6c716a7bcaf032fea1aec5caa2809de3b91b163b7"}
Oct 30 22:11:37.695143 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.695116    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz" event={"ID":"2eebc209-0c68-4e4c-9485-466eb87d6154","Type":"ContainerDied","Data":"b6568755565d57aff2df8536aa31eb88a2ecbde8ff5460dbe876c0941c168d32"}
Oct 30 22:11:37.704314 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.704296    5586 scope.go:122] "RemoveContainer" containerID="affa215878658387953b994491a29506ff0ada3fe72211fe8501492bf60d4838"
Oct 30 22:11:37.704583 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:11:37.704565    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"affa215878658387953b994491a29506ff0ada3fe72211fe8501492bf60d4838\": container with ID starting with affa215878658387953b994491a29506ff0ada3fe72211fe8501492bf60d4838 not found: ID does not exist" containerID="affa215878658387953b994491a29506ff0ada3fe72211fe8501492bf60d4838"
Oct 30 22:11:37.704627 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.704590    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affa215878658387953b994491a29506ff0ada3fe72211fe8501492bf60d4838"} err="failed to get container status \"affa215878658387953b994491a29506ff0ada3fe72211fe8501492bf60d4838\": rpc error: code = NotFound desc = could not find container \"affa215878658387953b994491a29506ff0ada3fe72211fe8501492bf60d4838\": container with ID starting with affa215878658387953b994491a29506ff0ada3fe72211fe8501492bf60d4838 not found: ID does not exist"
Oct 30 22:11:37.704627 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.704611    5586 scope.go:122] "RemoveContainer" containerID="e23d8d6f66078da4de7aa5c6c716a7bcaf032fea1aec5caa2809de3b91b163b7"
Oct 30 22:11:37.713771 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.713752    5586 scope.go:122] "RemoveContainer" containerID="e23d8d6f66078da4de7aa5c6c716a7bcaf032fea1aec5caa2809de3b91b163b7"
Oct 30 22:11:37.714040 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:11:37.714019    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e23d8d6f66078da4de7aa5c6c716a7bcaf032fea1aec5caa2809de3b91b163b7\": container with ID starting with e23d8d6f66078da4de7aa5c6c716a7bcaf032fea1aec5caa2809de3b91b163b7 not found: ID does not exist" containerID="e23d8d6f66078da4de7aa5c6c716a7bcaf032fea1aec5caa2809de3b91b163b7"
Oct 30 22:11:37.714094 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.714049    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e23d8d6f66078da4de7aa5c6c716a7bcaf032fea1aec5caa2809de3b91b163b7"} err="failed to get container status \"e23d8d6f66078da4de7aa5c6c716a7bcaf032fea1aec5caa2809de3b91b163b7\": rpc error: code = NotFound desc = could not find container \"e23d8d6f66078da4de7aa5c6c716a7bcaf032fea1aec5caa2809de3b91b163b7\": container with ID starting with e23d8d6f66078da4de7aa5c6c716a7bcaf032fea1aec5caa2809de3b91b163b7 not found: ID does not exist"
Oct 30 22:11:37.858842 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.858804    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"]
Oct 30 22:11:37.907711 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.907669    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/distinct-2-w5-9b51e1-kserve-75cbcbc448-qgrmz"]
Oct 30 22:11:37.968385 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.968294    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"]
Oct 30 22:11:37.971043 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.971020    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp"]
Oct 30 22:11:37.992475 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:37.992434    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9b51e1/distinct-w5-9b51e1-kserve-7ff97665dc-mqlpp" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Oct 30 22:11:38.082387 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.082353    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:38.286907 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.286816    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:38.354162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354126    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"]
Oct 30 22:11:38.354755 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354736    5586 cpu_manager.go:414] "containerMap: removing container" podUID="5a469237-ca99-4126-938b-bcbee6d9835f" containerName="maas-api"
Oct 30 22:11:38.354755 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354755    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a469237-ca99-4126-938b-bcbee6d9835f" containerName="maas-api"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354769    5586 cpu_manager.go:414] "containerMap: removing container" podUID="5fca9942-5704-4346-8c4f-abe2c9bfefe2" containerName="maas-api"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354774    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fca9942-5704-4346-8c4f-abe2c9bfefe2" containerName="maas-api"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354787    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerName="istio-proxy"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354792    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerName="istio-proxy"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354799    5586 cpu_manager.go:414] "containerMap: removing container" podUID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerName="istio-proxy"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354805    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerName="istio-proxy"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354815    5586 cpu_manager.go:414] "containerMap: removing container" podUID="b9c7ab71-5d38-4489-95fe-960b27c7bf40" containerName="payload-pre-processing"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354820    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9c7ab71-5d38-4489-95fe-960b27c7bf40" containerName="payload-pre-processing"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354829    5586 cpu_manager.go:414] "containerMap: removing container" podUID="4ec4ead0-ee39-4c04-8512-716f6143b147" containerName="payload-pre-processing"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354835    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ec4ead0-ee39-4c04-8512-716f6143b147" containerName="payload-pre-processing"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354845    5586 cpu_manager.go:414] "containerMap: removing container" podUID="dd404a46-bc6b-4a63-bd7c-e09793a2e377" containerName="payload-processing"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354850    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd404a46-bc6b-4a63-bd7c-e09793a2e377" containerName="payload-processing"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354857    5586 cpu_manager.go:414] "containerMap: removing container" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354862    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main"
Oct 30 22:11:38.354864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354868    5586 cpu_manager.go:414] "containerMap: removing container" podUID="603aeab9-aac6-4ed9-8336-08f7c2acdd6c" containerName="maas-api"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354873    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="603aeab9-aac6-4ed9-8336-08f7c2acdd6c" containerName="maas-api"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354880    5586 cpu_manager.go:414] "containerMap: removing container" podUID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354884    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354890    5586 cpu_manager.go:414] "containerMap: removing container" podUID="7b8ea9ca-9405-4abf-852c-04a8289fb6ec" containerName="payload-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354894    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b8ea9ca-9405-4abf-852c-04a8289fb6ec" containerName="payload-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354903    5586 cpu_manager.go:414] "containerMap: removing container" podUID="6a144510-a70d-4553-8717-b048d64094b3" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354907    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a144510-a70d-4553-8717-b048d64094b3" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354916    5586 cpu_manager.go:414] "containerMap: removing container" podUID="dfeb8098-593b-437f-acb7-74348724dea2" containerName="payload-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354920    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfeb8098-593b-437f-acb7-74348724dea2" containerName="payload-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354925    5586 cpu_manager.go:414] "containerMap: removing container" podUID="61e86a1d-3e10-4207-a9a1-d7a72072684d" containerName="payload-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354929    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61e86a1d-3e10-4207-a9a1-d7a72072684d" containerName="payload-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354940    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d67ce59f-5a5b-449e-9fcb-1207b9e151d3" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354944    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d67ce59f-5a5b-449e-9fcb-1207b9e151d3" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354953    5586 cpu_manager.go:414] "containerMap: removing container" podUID="1ae3b993-2c09-41f3-bcc7-eec79cae6198" containerName="maas-api"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354957    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ae3b993-2c09-41f3-bcc7-eec79cae6198" containerName="maas-api"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354967    5586 cpu_manager.go:414] "containerMap: removing container" podUID="6a144510-a70d-4553-8717-b048d64094b3" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354972    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a144510-a70d-4553-8717-b048d64094b3" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354980    5586 cpu_manager.go:414] "containerMap: removing container" podUID="b73ddad4-9be5-4fa2-9691-013116c525f7" containerName="payload-pre-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354984    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b73ddad4-9be5-4fa2-9691-013116c525f7" containerName="payload-pre-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354991    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d6ec046a-92b3-4116-a7ed-dde35559ccaa" containerName="maas-api"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.354995    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6ec046a-92b3-4116-a7ed-dde35559ccaa" containerName="maas-api"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355001    5586 cpu_manager.go:414] "containerMap: removing container" podUID="89b841c7-c736-4ffe-b31e-762908d62361" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355005    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89b841c7-c736-4ffe-b31e-762908d62361" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355012    5586 cpu_manager.go:414] "containerMap: removing container" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355016    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355028    5586 cpu_manager.go:414] "containerMap: removing container" podUID="1e584555-1a5c-429b-81b2-be625654d315" containerName="maas-api"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355032    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e584555-1a5c-429b-81b2-be625654d315" containerName="maas-api"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355039    5586 cpu_manager.go:414] "containerMap: removing container" podUID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355044    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355057    5586 cpu_manager.go:414] "containerMap: removing container" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355061    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355067    5586 cpu_manager.go:414] "containerMap: removing container" podUID="59f48521-b5bd-4e36-8200-7129da431734" containerName="payload-pre-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355073    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59f48521-b5bd-4e36-8200-7129da431734" containerName="payload-pre-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355079    5586 cpu_manager.go:414] "containerMap: removing container" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355084    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355091    5586 cpu_manager.go:414] "containerMap: removing container" podUID="89b841c7-c736-4ffe-b31e-762908d62361" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355094    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89b841c7-c736-4ffe-b31e-762908d62361" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355103    5586 cpu_manager.go:414] "containerMap: removing container" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355107    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355180    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9c7ab71-5d38-4489-95fe-960b27c7bf40" containerName="payload-pre-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355189    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ec4ead0-ee39-4c04-8512-716f6143b147" containerName="payload-pre-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355196    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd404a46-bc6b-4a63-bd7c-e09793a2e377" containerName="payload-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355200    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" containerName="main"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355204    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="603aeab9-aac6-4ed9-8336-08f7c2acdd6c" containerName="maas-api"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355211    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355217    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b8ea9ca-9405-4abf-852c-04a8289fb6ec" containerName="payload-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355223    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a144510-a70d-4553-8717-b048d64094b3" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355241    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="dfeb8098-593b-437f-acb7-74348724dea2" containerName="payload-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355247    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="61e86a1d-3e10-4207-a9a1-d7a72072684d" containerName="payload-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355254    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d67ce59f-5a5b-449e-9fcb-1207b9e151d3" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355259    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ae3b993-2c09-41f3-bcc7-eec79cae6198" containerName="maas-api"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355264    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a144510-a70d-4553-8717-b048d64094b3" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355269    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355274    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="b73ddad4-9be5-4fa2-9691-013116c525f7" containerName="payload-pre-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355278    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6ec046a-92b3-4116-a7ed-dde35559ccaa" containerName="maas-api"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355284    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="89b841c7-c736-4ffe-b31e-762908d62361" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355289    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355296    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e584555-1a5c-429b-81b2-be625654d315" containerName="maas-api"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355301    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="c20087fb-273e-4ba6-bc41-3a31cfeb30ec" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355306    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="0546e6b9-2621-4176-a47f-1bdd9e4c4ceb" containerName="main"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355311    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="59f48521-b5bd-4e36-8200-7129da431734" containerName="payload-pre-processing"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355317    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1307e68-d9b4-48e0-bad6-44854d56a0c4" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355322    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="89b841c7-c736-4ffe-b31e-762908d62361" containerName="istio-proxy"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355330    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" containerName="main"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355336    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a469237-ca99-4126-938b-bcbee6d9835f" containerName="maas-api"
Oct 30 22:11:38.355324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.355341    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fca9942-5704-4346-8c4f-abe2c9bfefe2" containerName="maas-api"
Oct 30 22:11:38.359854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.359837    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.368874 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.368847    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"]
Oct 30 22:11:38.369763 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.369741    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerName="istio-proxy"
Oct 30 22:11:38.369763 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.369762    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerName="istio-proxy"
Oct 30 22:11:38.369937 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.369895    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="a76b490b-c349-4a5c-ba7e-6445596e7ee0" containerName="istio-proxy"
Oct 30 22:11:38.369937 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.369912    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="aae92a23-65e7-4f07-bbf8-934cc18e916e" containerName="istio-proxy"
Oct 30 22:11:38.376945 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.375250    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.436423 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.425226    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"]
Oct 30 22:11:38.439215 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439175    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.439215 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439269    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.439215 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439312    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.439215 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439354    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.439215 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439378    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx789\" (UniqueName: \"kubernetes.io/projected/e8c9a965-426c-4d2c-bffb-3752255ac87c-kube-api-access-qx789\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.440380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439403    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.440380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439427    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.440380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439451    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c9a965-426c-4d2c-bffb-3752255ac87c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.440380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439473    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.440380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439504    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h799n\" (UniqueName: \"kubernetes.io/projected/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-kube-api-access-h799n\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.440380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439540    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.440380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439564    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.440380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439595    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.440380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439623    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.440380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439705    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.440380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439749    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.440380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439787    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.440380 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.439842    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.457260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.457212    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"]
Oct 30 22:11:38.541338 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541250    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.541338 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541302    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.541338 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541321    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx789\" (UniqueName: \"kubernetes.io/projected/e8c9a965-426c-4d2c-bffb-3752255ac87c-kube-api-access-qx789\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.541633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541435    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.541633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541469    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.541633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541494    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c9a965-426c-4d2c-bffb-3752255ac87c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.541633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541512    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.541633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541551    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h799n\" (UniqueName: \"kubernetes.io/projected/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-kube-api-access-h799n\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.541633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541599    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.541633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541623    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.541975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541655    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.541975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541687    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.541975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541784    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.541975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541794    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.541975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541838    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.541975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541879    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.541975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541902    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.541975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.541940    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.542429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.542005    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.542429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.542044    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.542429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.542082    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.542429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.542180    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.542429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.542246    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c9a965-426c-4d2c-bffb-3752255ac87c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.542996 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.542931    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.543112 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.543034    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.543457 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.543410    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.543667 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.543645    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.543756 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.543738    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.544653 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.544626    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.544827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.544804    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.545026 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.545006    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.545420 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.545404    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.597979 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.597944    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.598226 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.598210    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h799n\" (UniqueName: \"kubernetes.io/projected/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-kube-api-access-h799n\") pod \"maas-default-gateway-openshift-default-74854d8ccf-vxnxc\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:38.598322 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.598298    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.598430 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.598416    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx789\" (UniqueName: \"kubernetes.io/projected/e8c9a965-426c-4d2c-bffb-3752255ac87c-kube-api-access-qx789\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f2rlm\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.679468 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.679432    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:38.691481 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:38.691214    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:39.115982 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.115954    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"]
Oct 30 22:11:39.119960 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:11:39.119867    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c9a965_426c_4d2c_bffb_3752255ac87c.slice/crio-2a72054b89178bb7e03a8a4a233dce7baf268ad5d3769afee1e30c412caa2584 WatchSource:0}: Error finding container 2a72054b89178bb7e03a8a4a233dce7baf268ad5d3769afee1e30c412caa2584: Status 404 returned error can't find the container with id 2a72054b89178bb7e03a8a4a233dce7baf268ad5d3769afee1e30c412caa2584
Oct 30 22:11:39.121510 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.120614    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"]
Oct 30 22:11:39.128641 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.128565    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:39.129041 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.128984    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:39.129463 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.129441    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:39.189668 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.189622    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:39.189823 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.189719    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:39.189823 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.189763    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:11:39.463338 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.463270    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2eebc209-0c68-4e4c-9485-466eb87d6154" path="/var/lib/kubelet/pods/2eebc209-0c68-4e4c-9485-466eb87d6154/volumes"
Oct 30 22:11:39.464093 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.464064    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e412c89-24a0-474e-9e20-6be984b88f3e" path="/var/lib/kubelet/pods/6e412c89-24a0-474e-9e20-6be984b88f3e/volumes"
Oct 30 22:11:39.710989 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.710950    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:39.716446 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.716356    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm" event={"ID":"e8c9a965-426c-4d2c-bffb-3752255ac87c","Type":"ContainerStarted","Data":"b666d0ca1fdcc4f2a9148bc21e33676687d62eb91518555616ec03ece2197cfc"}
Oct 30 22:11:39.716446 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.716428    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm" event={"ID":"e8c9a965-426c-4d2c-bffb-3752255ac87c","Type":"ContainerStarted","Data":"2a72054b89178bb7e03a8a4a233dce7baf268ad5d3769afee1e30c412caa2584"}
Oct 30 22:11:39.718474 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.718442    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc" event={"ID":"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac","Type":"ContainerStarted","Data":"fe2697f6f4d48cfff91e10b97c14d1f1d85ed3908b1063018e2bcef9f077ceef"}
Oct 30 22:11:39.718645 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.718498    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc" event={"ID":"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac","Type":"ContainerStarted","Data":"9a8f823c4a6979cb62f466ba92864469b0c75676226e302dc3cf9ba8b235a91a"}
Oct 30 22:11:39.836256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.835299    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm" podStartSLOduration=1.8352778220000001 podStartE2EDuration="1.835277822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:39.832611102 +0000 UTC m=+2242.504515039" watchObservedRunningTime="2026-10-30 22:11:39.835277822 +0000 UTC m=+2242.507181757"
Oct 30 22:11:39.965259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:39.963696    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc" podStartSLOduration=1.96367182 podStartE2EDuration="1.96367182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:39.949761448 +0000 UTC m=+2242.621665384" watchObservedRunningTime="2026-10-30 22:11:39.96367182 +0000 UTC m=+2242.635575756"
Oct 30 22:11:40.683771 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:40.683410    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:40.695946 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:40.695903    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:11:40.697547 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:40.696296    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm" podUID="e8c9a965-426c-4d2c-bffb-3752255ac87c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 30 22:11:40.697717 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:40.697010    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:40.703365 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:40.703339    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 22:11:40.703568 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:40.703549    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc" podUID="012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 22:11:41.680613 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:41.680581    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:11:41.680866 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:41.680837    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm" podUID="e8c9a965-426c-4d2c-bffb-3752255ac87c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 30 22:11:41.691938 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:41.691902    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 22:11:41.692316 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:41.691953    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc" podUID="012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 22:11:41.718791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:41.718754    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:41.887992 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:41.887949    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:42.107269 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:42.106561    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"]
Oct 30 22:11:42.107269 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:42.106773    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx" podUID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerName="istio-proxy" containerID="cri-o://1d7a06da2d3873c2d13d528d0787be2f99d9074842b56170b972c53a370d9158" gracePeriod=30
Oct 30 22:11:42.107633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:42.107582    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:42.680986 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:42.680949    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:11:42.681170 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:42.681003    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm" podUID="e8c9a965-426c-4d2c-bffb-3752255ac87c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 30 22:11:42.692379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:42.692330    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 22:11:42.692732 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:42.692406    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc" podUID="012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 22:11:43.680759 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:43.680715    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:11:43.680926 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:43.680783    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm" podUID="e8c9a965-426c-4d2c-bffb-3752255ac87c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 30 22:11:43.692181 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:43.692144    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 22:11:43.692409 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:43.692209    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc" podUID="012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 22:11:44.047063 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:44.046959    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:44.274729 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:44.274695    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:44.680461 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:44.680424    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:11:44.680658 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:44.680474    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm" podUID="e8c9a965-426c-4d2c-bffb-3752255ac87c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 30 22:11:44.692845 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:44.692812    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 22:11:44.693156 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:44.692867    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc" podUID="012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 22:11:44.912911 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:44.912524    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:45.680331 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:45.680293    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:11:45.680530 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:45.680366    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm" podUID="e8c9a965-426c-4d2c-bffb-3752255ac87c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 30 22:11:45.692400 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:45.692367    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 22:11:45.692513 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:45.692423    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc" podUID="012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 22:11:46.680285 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:46.680228    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-f2rlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:11:46.680677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:46.680318    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm" podUID="e8c9a965-426c-4d2c-bffb-3752255ac87c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 30 22:11:46.692775 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:46.692742    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-vxnxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 22:11:46.692904 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:46.692803    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc" podUID="012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 22:11:47.440258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.439667    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:11:47.533038 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.532994    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-workload-certs\") pod \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") "
Oct 30 22:11:47.533246 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.533064    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-workload-socket\") pod \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") "
Oct 30 22:11:47.533246 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.533133    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-podinfo\") pod \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") "
Oct 30 22:11:47.533246 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.533217    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-data\") pod \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") "
Oct 30 22:11:47.533445 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.533273    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-credential-socket\") pod \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") "
Oct 30 22:11:47.533445 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.533317    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-token\" (UniqueName: \"kubernetes.io/projected/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-token\") pod \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") "
Oct 30 22:11:47.533445 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.533428    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-envoy\") pod \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") "
Oct 30 22:11:47.533587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.533500    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istiod-ca-cert\") pod \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") "
Oct 30 22:11:47.533587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.533569    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-kube-api-access-fwzp5\" (UniqueName: \"kubernetes.io/projected/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-kube-api-access-fwzp5\") pod \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\" (UID: \"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7\") "
Oct 30 22:11:47.535492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.535334    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-data" pod "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" (UID: "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:47.535492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.535260    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istiod-ca-cert" pod "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" (UID: "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:47.535724 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.535694    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-workload-socket" pod "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" (UID: "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:47.535724 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.535713    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-credential-socket" pod "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" (UID: "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:47.535892 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.535868    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-workload-certs" pod "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" (UID: "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:47.539068 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.539034    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-podinfo" pod "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" (UID: "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:47.539610 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.539586    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-envoy" pod "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" (UID: "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:47.540731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.540654    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-kube-api-access-fwzp5" pod "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" (UID: "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7"). InnerVolumeSpecName "kube-api-access-fwzp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:47.540731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.540701    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-token" pod "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" (UID: "51d056d9-bcc2-4af8-a50d-f46d2efd7eb7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:47.639392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.639200    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:47.639392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.639255    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:47.639392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.639271    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:47.639392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.639285    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:47.639392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.639302    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:47.639392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.639318    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:47.639392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.639334    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwzp5\" (UniqueName: \"kubernetes.io/projected/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-kube-api-access-fwzp5\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:47.639392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.639349    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:47.639392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.639363    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:47.844948 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.844898    5586 generic.go:356] "Generic (PLEG): container finished" podID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerID="1d7a06da2d3873c2d13d528d0787be2f99d9074842b56170b972c53a370d9158" exitCode=0
Oct 30 22:11:47.845509 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.844949    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx" event={"ID":"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7","Type":"ContainerDied","Data":"1d7a06da2d3873c2d13d528d0787be2f99d9074842b56170b972c53a370d9158"}
Oct 30 22:11:47.845509 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.845053    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx" event={"ID":"51d056d9-bcc2-4af8-a50d-f46d2efd7eb7","Type":"ContainerDied","Data":"f2d9979076f29f3ffc2d2219e13e996516270a5ad9cb228061458a07285e4f7d"}
Oct 30 22:11:47.845509 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.845089    5586 scope.go:122] "RemoveContainer" containerID="1d7a06da2d3873c2d13d528d0787be2f99d9074842b56170b972c53a370d9158"
Oct 30 22:11:47.845509 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.845496    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"
Oct 30 22:11:47.869333 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.868468    5586 scope.go:122] "RemoveContainer" containerID="183b7260b4d2ee4c0c91cde2927605d1dbe4d7065288829b79369a7bc3ad52f6"
Oct 30 22:11:47.883606 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.883573    5586 scope.go:122] "RemoveContainer" containerID="1d7a06da2d3873c2d13d528d0787be2f99d9074842b56170b972c53a370d9158"
Oct 30 22:11:47.884008 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:11:47.883974    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d7a06da2d3873c2d13d528d0787be2f99d9074842b56170b972c53a370d9158\": container with ID starting with 1d7a06da2d3873c2d13d528d0787be2f99d9074842b56170b972c53a370d9158 not found: ID does not exist" containerID="1d7a06da2d3873c2d13d528d0787be2f99d9074842b56170b972c53a370d9158"
Oct 30 22:11:47.884126 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.884021    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d7a06da2d3873c2d13d528d0787be2f99d9074842b56170b972c53a370d9158"} err="failed to get container status \"1d7a06da2d3873c2d13d528d0787be2f99d9074842b56170b972c53a370d9158\": rpc error: code = NotFound desc = could not find container \"1d7a06da2d3873c2d13d528d0787be2f99d9074842b56170b972c53a370d9158\": container with ID starting with 1d7a06da2d3873c2d13d528d0787be2f99d9074842b56170b972c53a370d9158 not found: ID does not exist"
Oct 30 22:11:47.884126 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.884047    5586 scope.go:122] "RemoveContainer" containerID="183b7260b4d2ee4c0c91cde2927605d1dbe4d7065288829b79369a7bc3ad52f6"
Oct 30 22:11:47.884486 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:11:47.884454    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"183b7260b4d2ee4c0c91cde2927605d1dbe4d7065288829b79369a7bc3ad52f6\": container with ID starting with 183b7260b4d2ee4c0c91cde2927605d1dbe4d7065288829b79369a7bc3ad52f6 not found: ID does not exist" containerID="183b7260b4d2ee4c0c91cde2927605d1dbe4d7065288829b79369a7bc3ad52f6"
Oct 30 22:11:47.884603 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:47.884496    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"183b7260b4d2ee4c0c91cde2927605d1dbe4d7065288829b79369a7bc3ad52f6"} err="failed to get container status \"183b7260b4d2ee4c0c91cde2927605d1dbe4d7065288829b79369a7bc3ad52f6\": rpc error: code = NotFound desc = could not find container \"183b7260b4d2ee4c0c91cde2927605d1dbe4d7065288829b79369a7bc3ad52f6\": container with ID starting with 183b7260b4d2ee4c0c91cde2927605d1dbe4d7065288829b79369a7bc3ad52f6 not found: ID does not exist"
Oct 30 22:11:48.013838 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:48.013801    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"]
Oct 30 22:11:48.018557 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:48.018519    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-1db768-openshift-default-85c476bb45-46vrx"]
Oct 30 22:11:48.286025 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:48.285984    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:48.287111 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:48.287087    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:48.288337 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:48.288314    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:11:48.329376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:48.329343    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:48.329801 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:48.329772    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:48.330769 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:48.330751    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:11:48.363079 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:11:48.363044    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:11:49.109887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:49.109837    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:49.314384 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:49.314346    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:49.456425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:49.454989    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" path="/var/lib/kubelet/pods/51d056d9-bcc2-4af8-a50d-f46d2efd7eb7/volumes"
Oct 30 22:11:49.604766 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:49.604727    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:52.363939 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:52.363896    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:53.036938 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:53.036897    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:53.192882 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:53.192843    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:54.217795 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.217762    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"]
Oct 30 22:11:54.218357 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.218342    5586 cpu_manager.go:414] "containerMap: removing container" podUID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerName="istio-proxy"
Oct 30 22:11:54.218407 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.218358    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerName="istio-proxy"
Oct 30 22:11:54.218407 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.218385    5586 cpu_manager.go:414] "containerMap: removing container" podUID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerName="istio-proxy"
Oct 30 22:11:54.218407 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.218391    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerName="istio-proxy"
Oct 30 22:11:54.218525 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.218506    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerName="istio-proxy"
Oct 30 22:11:54.218525 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.218517    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="51d056d9-bcc2-4af8-a50d-f46d2efd7eb7" containerName="istio-proxy"
Oct 30 22:11:54.223448 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.223421    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:54.326079 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.326048    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6df3ba0e-serving-cert\""
Oct 30 22:11:54.329784 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.329757    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"]
Oct 30 22:11:54.334566 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.334544    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-maas-api-tls\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:54.334687 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.334577    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg52h\" (UniqueName: \"kubernetes.io/projected/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-kube-api-access-qg52h\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:54.334687 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.334596    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-metrics-certs\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:54.334775 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.334751    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-openshift-service-ca\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:54.422529 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.422496    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:54.435728 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.435698    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-maas-api-tls\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:54.435872 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.435737    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg52h\" (UniqueName: \"kubernetes.io/projected/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-kube-api-access-qg52h\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:54.435872 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.435762    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-metrics-certs\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:54.435872 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.435806    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-openshift-service-ca\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:54.436071 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:11:54.436045    5586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-serving-cert: secret "maas-api-e2e-mt-6df3ba0e-serving-cert" not found
Oct 30 22:11:54.436125 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:11:54.436115    5586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-maas-api-tls podName:5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59 nodeName:}" failed. No retries permitted until 2026-10-30 22:11:54.936100008 +0000 UTC m=+2257.608003934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-maas-api-tls") pod "maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc" (UID: "5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59") : secret "maas-api-e2e-mt-6df3ba0e-serving-cert" not found
Oct 30 22:11:54.436576 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.436556    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-openshift-service-ca\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:54.438320 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.438300    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-metrics-certs\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:54.494715 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.494632    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg52h\" (UniqueName: \"kubernetes.io/projected/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-kube-api-access-qg52h\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:54.585172 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.585131    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:54.739979 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.739928    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:54.941449 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.941405    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-maas-api-tls\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:54.944037 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:54.944006    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-maas-api-tls\") pod \"maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:55.137715 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:55.137679    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:55.517837 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:55.517808    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"]
Oct 30 22:11:55.520022 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:11:55.519997    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c1c2bf2_e30f_41c9_ad29_0c1ab6960d59.slice/crio-56e8d7101f2c3a267ae70b56837834885a8e121c32334f2ed24e5e8febbc66c5 WatchSource:0}: Error finding container 56e8d7101f2c3a267ae70b56837834885a8e121c32334f2ed24e5e8febbc66c5: Status 404 returned error can't find the container with id 56e8d7101f2c3a267ae70b56837834885a8e121c32334f2ed24e5e8febbc66c5
Oct 30 22:11:55.521820 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:55.521798    5586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:11:55.906127 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:55.906089    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc" event={"ID":"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59","Type":"ContainerStarted","Data":"56e8d7101f2c3a267ae70b56837834885a8e121c32334f2ed24e5e8febbc66c5"}
Oct 30 22:11:56.916577 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:56.916537    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc" event={"ID":"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59","Type":"ContainerStarted","Data":"ce91548e25be14d637040bfef9e06dbb872fc7b35d5c19b9b647d71fe2fc8980"}
Oct 30 22:11:56.917041 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:56.916616    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:11:57.034804 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:57.034744    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc" podStartSLOduration=2.233119812 podStartE2EDuration="3.034721061s" totalImagesPullingTime="801.601249ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:57.030986631 +0000 UTC m=+2259.702890624" watchObservedRunningTime="2026-10-30 22:11:57.034721061 +0000 UTC m=+2259.706624997"
Oct 30 22:11:58.597156 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:58.597074    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:58.804517 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:58.804480    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:11:59.380801 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:11:59.380756    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:00.301583 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:00.301546    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:00.479898 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:00.479866    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:02.930440 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:02.930404    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:12:03.920180 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:03.920147    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:04.172291 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:04.172162    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:04.334609 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:04.334572    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:06.854516 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:06.854423    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:09.233185 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:09.233129    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:13.930278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:13.930225    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:18.361121 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:12:18.361072    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:12:18.706566 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:18.706523    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:23.489401 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:23.489347    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:25.641883 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:25.641844    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:27.917573 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:27.917526    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"]
Oct 30 22:12:27.918116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:27.917767    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4" podUID="5deff068-24f9-412b-a6ec-f3d106d24002" containerName="maas-api" containerID="cri-o://6af4467d62c1b410e5e8d1ef4c7871535f4b7bca7016f6a546f0f2c596fcf3c7" gracePeriod=30
Oct 30 22:12:28.067117 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.067067    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:28.228629 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.228591    5586 generic.go:356] "Generic (PLEG): container finished" podID="5deff068-24f9-412b-a6ec-f3d106d24002" containerID="6af4467d62c1b410e5e8d1ef4c7871535f4b7bca7016f6a546f0f2c596fcf3c7" exitCode=0
Oct 30 22:12:28.228789 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.228660    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4" event={"ID":"5deff068-24f9-412b-a6ec-f3d106d24002","Type":"ContainerDied","Data":"6af4467d62c1b410e5e8d1ef4c7871535f4b7bca7016f6a546f0f2c596fcf3c7"}
Oct 30 22:12:28.228789 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.228674    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:12:28.228789 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.228755    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4" event={"ID":"5deff068-24f9-412b-a6ec-f3d106d24002","Type":"ContainerDied","Data":"589760ad9b9e07fe3b8893f7ccde902504d105de3d855125914e8e2772121417"}
Oct 30 22:12:28.228789 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.228776    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="589760ad9b9e07fe3b8893f7ccde902504d105de3d855125914e8e2772121417"
Oct 30 22:12:28.324343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.324306    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-maas-api-tls\") pod \"5deff068-24f9-412b-a6ec-f3d106d24002\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") "
Oct 30 22:12:28.324523 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.324471    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-metrics-certs\") pod \"5deff068-24f9-412b-a6ec-f3d106d24002\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") "
Oct 30 22:12:28.324523 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.324504    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5deff068-24f9-412b-a6ec-f3d106d24002-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5deff068-24f9-412b-a6ec-f3d106d24002-openshift-service-ca\") pod \"5deff068-24f9-412b-a6ec-f3d106d24002\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") "
Oct 30 22:12:28.324616 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.324548    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5deff068-24f9-412b-a6ec-f3d106d24002-kube-api-access-2xl7d\" (UniqueName: \"kubernetes.io/projected/5deff068-24f9-412b-a6ec-f3d106d24002-kube-api-access-2xl7d\") pod \"5deff068-24f9-412b-a6ec-f3d106d24002\" (UID: \"5deff068-24f9-412b-a6ec-f3d106d24002\") "
Oct 30 22:12:28.324930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.324896    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5deff068-24f9-412b-a6ec-f3d106d24002-openshift-service-ca" pod "5deff068-24f9-412b-a6ec-f3d106d24002" (UID: "5deff068-24f9-412b-a6ec-f3d106d24002"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:12:28.325274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.325251    5586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5deff068-24f9-412b-a6ec-f3d106d24002-openshift-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:28.337974 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.337879    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5deff068-24f9-412b-a6ec-f3d106d24002-kube-api-access-2xl7d" pod "5deff068-24f9-412b-a6ec-f3d106d24002" (UID: "5deff068-24f9-412b-a6ec-f3d106d24002"). InnerVolumeSpecName "kube-api-access-2xl7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:28.339942 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.339893    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-maas-api-tls" pod "5deff068-24f9-412b-a6ec-f3d106d24002" (UID: "5deff068-24f9-412b-a6ec-f3d106d24002"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:12:28.341551 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.340903    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-metrics-certs" pod "5deff068-24f9-412b-a6ec-f3d106d24002" (UID: "5deff068-24f9-412b-a6ec-f3d106d24002"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:12:28.426906 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.426866    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-maas-api-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:28.426906 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.426899    5586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5deff068-24f9-412b-a6ec-f3d106d24002-metrics-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:28.426906 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.426911    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xl7d\" (UniqueName: \"kubernetes.io/projected/5deff068-24f9-412b-a6ec-f3d106d24002-kube-api-access-2xl7d\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:28.658914 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:28.658862    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:29.235291 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:29.235256    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"
Oct 30 22:12:29.400860 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:29.400821    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"]
Oct 30 22:12:29.450157 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:29.450126    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9f8e92-75f4d4d95d-pf7d4"]
Oct 30 22:12:30.335617 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:30.335575    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:30.576224 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:30.576189    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:31.448024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:31.447989    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5deff068-24f9-412b-a6ec-f3d106d24002" path="/var/lib/kubelet/pods/5deff068-24f9-412b-a6ec-f3d106d24002/volumes"
Oct 30 22:12:31.839282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:31.839174    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:33.217712 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:33.217670    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:35.272802 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:35.272759    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:36.786537 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:36.786494    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:37.015988 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:37.015941    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:37.155135 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:37.155098    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"]
Oct 30 22:12:37.155406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:37.155378    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw" podUID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerName="istio-proxy" containerID="cri-o://8ef13f21394fa37fe8519023b6e70613b2ee6beb1dd730df47293133d7a0db10" gracePeriod=30
Oct 30 22:12:37.203977 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:37.203934    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:37.379648 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:37.379609    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:38.686968 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.686925    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"]
Oct 30 22:12:38.687810 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.687786    5586 cpu_manager.go:414] "containerMap: removing container" podUID="5deff068-24f9-412b-a6ec-f3d106d24002" containerName="maas-api"
Oct 30 22:12:38.687810 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.687811    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5deff068-24f9-412b-a6ec-f3d106d24002" containerName="maas-api"
Oct 30 22:12:38.688007 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.687993    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="5deff068-24f9-412b-a6ec-f3d106d24002" containerName="maas-api"
Oct 30 22:12:38.692767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.692734    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.745710 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.745672    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-954628-openshift-default-dockercfg-vmcxz\""
Oct 30 22:12:38.762808 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.762779    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26432f05-beba-4093-a139-217dac85db7e-istiod-ca-cert\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.762948 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.762816    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-workload-certs\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.762948 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.762897    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26432f05-beba-4093-a139-217dac85db7e-istio-token\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.763114 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.763062    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-istio-envoy\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.763114 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.763099    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26432f05-beba-4093-a139-217dac85db7e-istio-podinfo\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.763265 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.763216    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-istio-data\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.763319 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.763279    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-credential-socket\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.763389 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.763369    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-workload-socket\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.763432 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.763402    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn257\" (UniqueName: \"kubernetes.io/projected/26432f05-beba-4093-a139-217dac85db7e-kube-api-access-nn257\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.781961 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.781928    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:38.812437 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.812398    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"]
Oct 30 22:12:38.865044 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.865009    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26432f05-beba-4093-a139-217dac85db7e-istiod-ca-cert\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.865044 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.865050    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-workload-certs\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.865373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.865069    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26432f05-beba-4093-a139-217dac85db7e-istio-token\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.865373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.865139    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-istio-envoy\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.865373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.865273    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26432f05-beba-4093-a139-217dac85db7e-istio-podinfo\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.865373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.865357    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-istio-data\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.865604 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.865417    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-credential-socket\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.865604 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.865488    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-workload-socket\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.865604 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.865517    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn257\" (UniqueName: \"kubernetes.io/projected/26432f05-beba-4093-a139-217dac85db7e-kube-api-access-nn257\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.865768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.865669    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-workload-certs\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.865857 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.865828    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-istio-data\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.865927 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.865862    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26432f05-beba-4093-a139-217dac85db7e-istiod-ca-cert\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.866071 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.866047    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-workload-socket\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.866149 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.866091    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-credential-socket\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.867768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.867741    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26432f05-beba-4093-a139-217dac85db7e-istio-podinfo\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.867880 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.867806    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-istio-envoy\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.938100 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.938031    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26432f05-beba-4093-a139-217dac85db7e-istio-token\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:38.939363 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:38.939341    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn257\" (UniqueName: \"kubernetes.io/projected/26432f05-beba-4093-a139-217dac85db7e-kube-api-access-nn257\") pod \"e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") " pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:39.007541 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:39.007505    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:39.009374 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:39.009350    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:39.450373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:39.450343    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"]
Oct 30 22:12:39.451669 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:12:39.451641    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26432f05_beba_4093_a139_217dac85db7e.slice/crio-b07e36371ae7b7f8edb971b0fae871e097852555584cc68d855ef17dbad64a73 WatchSource:0}: Error finding container b07e36371ae7b7f8edb971b0fae871e097852555584cc68d855ef17dbad64a73: Status 404 returned error can't find the container with id b07e36371ae7b7f8edb971b0fae871e097852555584cc68d855ef17dbad64a73
Oct 30 22:12:39.454172 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:39.454140    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:12:39.454260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:39.454207    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:12:39.454260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:39.454254    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:12:40.332847 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:40.332810    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs" event={"ID":"26432f05-beba-4093-a139-217dac85db7e","Type":"ContainerStarted","Data":"82b93ee8f4e0657f07b778d752df9415be6df278addfdfb9a327430e339434cb"}
Oct 30 22:12:40.333349 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:40.332870    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs" event={"ID":"26432f05-beba-4093-a139-217dac85db7e","Type":"ContainerStarted","Data":"b07e36371ae7b7f8edb971b0fae871e097852555584cc68d855ef17dbad64a73"}
Oct 30 22:12:40.507876 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:40.507820    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs" podStartSLOduration=2.507804674 podStartE2EDuration="2.507804674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:40.505727496 +0000 UTC m=+2303.177631431" watchObservedRunningTime="2026-10-30 22:12:40.507804674 +0000 UTC m=+2303.179708608"
Oct 30 22:12:40.638354 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:40.638267    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:41.009804 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:41.009768    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:41.015918 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:41.015889    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:41.340554 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:41.340460    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:41.341747 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:41.341722    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:12:42.363884 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.363845    5586 generic.go:356] "Generic (PLEG): container finished" podID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerID="8ef13f21394fa37fe8519023b6e70613b2ee6beb1dd730df47293133d7a0db10" exitCode=0
Oct 30 22:12:42.364397 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.363921    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw" event={"ID":"02b0df12-ffa4-4f62-b12d-fd1e2808a730","Type":"ContainerDied","Data":"8ef13f21394fa37fe8519023b6e70613b2ee6beb1dd730df47293133d7a0db10"}
Oct 30 22:12:42.364397 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.364002    5586 scope.go:122] "RemoveContainer" containerID="7b783f94d3d7052180811ae2d6114920f2987a6e2c6aec2c0eba42d09fde7c2c"
Oct 30 22:12:42.439449 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.439420    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:12:42.513743 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.513705    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-podinfo\") pod \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") "
Oct 30 22:12:42.513955 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.513755    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02b0df12-ffa4-4f62-b12d-fd1e2808a730-kube-api-access-zrln4\" (UniqueName: \"kubernetes.io/projected/02b0df12-ffa4-4f62-b12d-fd1e2808a730-kube-api-access-zrln4\") pod \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") "
Oct 30 22:12:42.513955 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.513802    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-token\" (UniqueName: \"kubernetes.io/projected/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-token\") pod \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") "
Oct 30 22:12:42.513955 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.513871    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-credential-socket\") pod \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") "
Oct 30 22:12:42.513955 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.513946    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-workload-socket\") pod \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") "
Oct 30 22:12:42.514174 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.513991    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-data\") pod \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") "
Oct 30 22:12:42.514174 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.514075    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istiod-ca-cert\") pod \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") "
Oct 30 22:12:42.514312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.514191    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-credential-socket" pod "02b0df12-ffa4-4f62-b12d-fd1e2808a730" (UID: "02b0df12-ffa4-4f62-b12d-fd1e2808a730"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:42.514312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.514202    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-workload-certs\") pod \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") "
Oct 30 22:12:42.514435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.514337    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-envoy\") pod \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\" (UID: \"02b0df12-ffa4-4f62-b12d-fd1e2808a730\") "
Oct 30 22:12:42.514501 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.514449    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-data" pod "02b0df12-ffa4-4f62-b12d-fd1e2808a730" (UID: "02b0df12-ffa4-4f62-b12d-fd1e2808a730"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:42.514501 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.514465    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-workload-certs" pod "02b0df12-ffa4-4f62-b12d-fd1e2808a730" (UID: "02b0df12-ffa4-4f62-b12d-fd1e2808a730"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:42.514759 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.514731    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-workload-socket" pod "02b0df12-ffa4-4f62-b12d-fd1e2808a730" (UID: "02b0df12-ffa4-4f62-b12d-fd1e2808a730"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:42.515012 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.514981    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:42.515012 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.515012    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:42.515203 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.515028    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:42.515203 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.515041    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:42.515365 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.515314    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istiod-ca-cert" pod "02b0df12-ffa4-4f62-b12d-fd1e2808a730" (UID: "02b0df12-ffa4-4f62-b12d-fd1e2808a730"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:12:42.516384 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.516350    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-podinfo" pod "02b0df12-ffa4-4f62-b12d-fd1e2808a730" (UID: "02b0df12-ffa4-4f62-b12d-fd1e2808a730"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:12:42.516495 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.516470    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02b0df12-ffa4-4f62-b12d-fd1e2808a730-kube-api-access-zrln4" pod "02b0df12-ffa4-4f62-b12d-fd1e2808a730" (UID: "02b0df12-ffa4-4f62-b12d-fd1e2808a730"). InnerVolumeSpecName "kube-api-access-zrln4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:42.516603 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.516576    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-token" pod "02b0df12-ffa4-4f62-b12d-fd1e2808a730" (UID: "02b0df12-ffa4-4f62-b12d-fd1e2808a730"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:42.516971 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.516950    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-envoy" pod "02b0df12-ffa4-4f62-b12d-fd1e2808a730" (UID: "02b0df12-ffa4-4f62-b12d-fd1e2808a730"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:42.616258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.616040    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:42.616258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.616076    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:42.616258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.616092    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:42.616258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.616105    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02b0df12-ffa4-4f62-b12d-fd1e2808a730-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:42.616258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:42.616118    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrln4\" (UniqueName: \"kubernetes.io/projected/02b0df12-ffa4-4f62-b12d-fd1e2808a730-kube-api-access-zrln4\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:43.372847 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:43.372814    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"
Oct 30 22:12:43.373418 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:43.372849    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw" event={"ID":"02b0df12-ffa4-4f62-b12d-fd1e2808a730","Type":"ContainerDied","Data":"3661f2445f64da003c4dbbc87672486f7dd2d06c2e77984edcb3fca1b984f4e4"}
Oct 30 22:12:43.373418 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:43.372909    5586 scope.go:122] "RemoveContainer" containerID="8ef13f21394fa37fe8519023b6e70613b2ee6beb1dd730df47293133d7a0db10"
Oct 30 22:12:43.456498 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:43.456450    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:43.647316 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:43.641316    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"]
Oct 30 22:12:43.651888 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:43.651858    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-9f8e92-openshift-default-86856d5b7-pfkjw"]
Oct 30 22:12:43.748393 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:43.748357    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:44.197068 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:44.197030    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:45.455253 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:45.455188    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" path="/var/lib/kubelet/pods/02b0df12-ffa4-4f62-b12d-fd1e2808a730/volumes"
Oct 30 22:12:45.560553 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:45.560516    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"]
Oct 30 22:12:45.560796 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:45.560766    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx" podUID="6b85c642-f2d3-4fb1-9328-3dea529752de" containerName="maas-api" containerID="cri-o://8f56572cff08c4306612591c7fbc05a38ec653c060334baed9f59d2c8a84ea60" gracePeriod=30
Oct 30 22:12:45.702465 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:45.702421    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:45.902368 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:45.902341    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 22:12:45.986256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:45.984691    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-maas-api-tls\") pod \"6b85c642-f2d3-4fb1-9328-3dea529752de\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") "
Oct 30 22:12:45.986256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:45.984758    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b85c642-f2d3-4fb1-9328-3dea529752de-kube-api-access-r2zb2\" (UniqueName: \"kubernetes.io/projected/6b85c642-f2d3-4fb1-9328-3dea529752de-kube-api-access-r2zb2\") pod \"6b85c642-f2d3-4fb1-9328-3dea529752de\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") "
Oct 30 22:12:45.986256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:45.984860    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b85c642-f2d3-4fb1-9328-3dea529752de-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b85c642-f2d3-4fb1-9328-3dea529752de-openshift-service-ca\") pod \"6b85c642-f2d3-4fb1-9328-3dea529752de\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") "
Oct 30 22:12:45.986256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:45.984988    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-metrics-certs\") pod \"6b85c642-f2d3-4fb1-9328-3dea529752de\" (UID: \"6b85c642-f2d3-4fb1-9328-3dea529752de\") "
Oct 30 22:12:45.992653 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:45.992424    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-metrics-certs" pod "6b85c642-f2d3-4fb1-9328-3dea529752de" (UID: "6b85c642-f2d3-4fb1-9328-3dea529752de"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:12:45.993217 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:45.992953    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b85c642-f2d3-4fb1-9328-3dea529752de-openshift-service-ca" pod "6b85c642-f2d3-4fb1-9328-3dea529752de" (UID: "6b85c642-f2d3-4fb1-9328-3dea529752de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:12:46.004253 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.002473    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b85c642-f2d3-4fb1-9328-3dea529752de-kube-api-access-r2zb2" pod "6b85c642-f2d3-4fb1-9328-3dea529752de" (UID: "6b85c642-f2d3-4fb1-9328-3dea529752de"). InnerVolumeSpecName "kube-api-access-r2zb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:46.010341 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.010258    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-maas-api-tls" pod "6b85c642-f2d3-4fb1-9328-3dea529752de" (UID: "6b85c642-f2d3-4fb1-9328-3dea529752de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:12:46.086920 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.086881    5586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-metrics-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:46.086920 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.086911    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b85c642-f2d3-4fb1-9328-3dea529752de-maas-api-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:46.086920 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.086922    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2zb2\" (UniqueName: \"kubernetes.io/projected/6b85c642-f2d3-4fb1-9328-3dea529752de-kube-api-access-r2zb2\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:46.086920 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.086933    5586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b85c642-f2d3-4fb1-9328-3dea529752de-openshift-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:46.397089 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.396985    5586 generic.go:356] "Generic (PLEG): container finished" podID="6b85c642-f2d3-4fb1-9328-3dea529752de" containerID="8f56572cff08c4306612591c7fbc05a38ec653c060334baed9f59d2c8a84ea60" exitCode=0
Oct 30 22:12:46.397089 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.397045    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"
Oct 30 22:12:46.397089 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.397074    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx" event={"ID":"6b85c642-f2d3-4fb1-9328-3dea529752de","Type":"ContainerDied","Data":"8f56572cff08c4306612591c7fbc05a38ec653c060334baed9f59d2c8a84ea60"}
Oct 30 22:12:46.397386 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.397154    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx" event={"ID":"6b85c642-f2d3-4fb1-9328-3dea529752de","Type":"ContainerDied","Data":"5daa538e03fb0aa717a38aff1fe772c7606f0eb8a7d6df44912b35f25417f10a"}
Oct 30 22:12:46.397386 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.397177    5586 scope.go:122] "RemoveContainer" containerID="8f56572cff08c4306612591c7fbc05a38ec653c060334baed9f59d2c8a84ea60"
Oct 30 22:12:46.408990 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.408958    5586 scope.go:122] "RemoveContainer" containerID="8f56572cff08c4306612591c7fbc05a38ec653c060334baed9f59d2c8a84ea60"
Oct 30 22:12:46.409326 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:12:46.409305    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f56572cff08c4306612591c7fbc05a38ec653c060334baed9f59d2c8a84ea60\": container with ID starting with 8f56572cff08c4306612591c7fbc05a38ec653c060334baed9f59d2c8a84ea60 not found: ID does not exist" containerID="8f56572cff08c4306612591c7fbc05a38ec653c060334baed9f59d2c8a84ea60"
Oct 30 22:12:46.409398 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.409336    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f56572cff08c4306612591c7fbc05a38ec653c060334baed9f59d2c8a84ea60"} err="failed to get container status \"8f56572cff08c4306612591c7fbc05a38ec653c060334baed9f59d2c8a84ea60\": rpc error: code = NotFound desc = could not find container \"8f56572cff08c4306612591c7fbc05a38ec653c060334baed9f59d2c8a84ea60\": container with ID starting with 8f56572cff08c4306612591c7fbc05a38ec653c060334baed9f59d2c8a84ea60 not found: ID does not exist"
Oct 30 22:12:46.562138 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.562103    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"]
Oct 30 22:12:46.638687 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.628199    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dda9f5-747f54f74b-cgslx"]
Oct 30 22:12:46.732259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.717494    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:46.914895 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:46.914857    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:47.467642 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:47.467602    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b85c642-f2d3-4fb1-9328-3dea529752de" path="/var/lib/kubelet/pods/6b85c642-f2d3-4fb1-9328-3dea529752de/volumes"
Oct 30 22:12:47.549253 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:47.546159    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:47.753101 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:47.753011    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:47.917684 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:47.917641    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:48.382254 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:12:48.381520    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:12:48.544332 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:48.544288    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:48.869339 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:48.869286    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:49.034088 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:49.034050    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:53.031281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:53.031243    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:55.783051 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:55.782999    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:55.968100 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:55.968046    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:56.146436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:56.146336    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:57.488260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:57.484841    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:57.670113 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:57.670077    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:12:58.418243 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:12:58.418193    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:00.886469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:00.886435    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:01.058689 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:01.058649    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:02.490866 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:02.490824    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:03.597479 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:03.597445    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:08.511786 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:08.511748    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:08.999497 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:08.999459    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:10.905329 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.905294    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"]
Oct 30 22:13:10.905889 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.905875    5586 cpu_manager.go:414] "containerMap: removing container" podUID="6b85c642-f2d3-4fb1-9328-3dea529752de" containerName="maas-api"
Oct 30 22:13:10.905948 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.905890    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b85c642-f2d3-4fb1-9328-3dea529752de" containerName="maas-api"
Oct 30 22:13:10.905948 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.905911    5586 cpu_manager.go:414] "containerMap: removing container" podUID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerName="istio-proxy"
Oct 30 22:13:10.905948 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.905917    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerName="istio-proxy"
Oct 30 22:13:10.905948 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.905939    5586 cpu_manager.go:414] "containerMap: removing container" podUID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerName="istio-proxy"
Oct 30 22:13:10.905948 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.905946    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerName="istio-proxy"
Oct 30 22:13:10.906147 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.906048    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b85c642-f2d3-4fb1-9328-3dea529752de" containerName="maas-api"
Oct 30 22:13:10.906147 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.906058    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerName="istio-proxy"
Oct 30 22:13:10.906147 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.906069    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="02b0df12-ffa4-4f62-b12d-fd1e2808a730" containerName="istio-proxy"
Oct 30 22:13:10.910740 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.910721    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:10.968649 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.968604    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:10.968867 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.968730    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:10.968867 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.968779    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:10.968867 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.968801    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppzbd\" (UniqueName: \"kubernetes.io/projected/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-kube-api-access-ppzbd\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:10.968867 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.968818    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:10.969017 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.968871    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:10.969017 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.968960    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:10.969082 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.969025    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:10.969116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:10.969080    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.015930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.015892    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"]
Oct 30 22:13:11.070727 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.070688    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.070930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.070762    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.070930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.070801    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppzbd\" (UniqueName: \"kubernetes.io/projected/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-kube-api-access-ppzbd\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.070930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.070825    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.070930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.070849    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.070930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.070877    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.070930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.070904    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.071339 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.070951    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.071339 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.071113    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.071469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.071328    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.071603 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.071581    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.071765 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.071736    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.071904 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.071768    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.071978 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.071941    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.073596 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.073575    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.073805 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.073788    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.129965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.129930    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.130130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.129968    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppzbd\" (UniqueName: \"kubernetes.io/projected/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-kube-api-access-ppzbd\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.225072 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.224974    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:11.657533 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.657501    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"]
Oct 30 22:13:11.659059 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:13:11.659027    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc6b9f2d_596c_4c77_ae0a_ae38b52a64bf.slice/crio-b4a9ff65925e8076eeecf11cc25a9e12f9c39eb7d099809313b28c8a1baf4559 WatchSource:0}: Error finding container b4a9ff65925e8076eeecf11cc25a9e12f9c39eb7d099809313b28c8a1baf4559: Status 404 returned error can't find the container with id b4a9ff65925e8076eeecf11cc25a9e12f9c39eb7d099809313b28c8a1baf4559
Oct 30 22:13:11.661572 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.661536    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:11.661679 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.661611    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:11.661679 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:11.661639    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:12.609214 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:12.609168    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc" event={"ID":"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf","Type":"ContainerStarted","Data":"0329e768f94b56a32957dc17e7d72d027bed84ba947f026ba1c068b27d70a371"}
Oct 30 22:13:12.609747 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:12.609226    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc" event={"ID":"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf","Type":"ContainerStarted","Data":"b4a9ff65925e8076eeecf11cc25a9e12f9c39eb7d099809313b28c8a1baf4559"}
Oct 30 22:13:12.781947 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:12.781887    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc" podStartSLOduration=2.781870206 podStartE2EDuration="2.781870206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:12.779584392 +0000 UTC m=+2335.451488319" watchObservedRunningTime="2026-10-30 22:13:12.781870206 +0000 UTC m=+2335.453774141"
Oct 30 22:13:13.232258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:13.228645    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:13.246619 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:13.246430    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 30 22:13:13.246619 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:13.246538    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc" podUID="dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 30 22:13:13.695938 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:13.695901    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:14.246291 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:14.245717    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 30 22:13:14.246291 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:14.245782    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc" podUID="dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 30 22:13:15.236260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:15.227702    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 30 22:13:15.236260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:15.227768    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc" podUID="dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 30 22:13:16.227688 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:16.227648    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 30 22:13:16.228018 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:16.227987    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc" podUID="dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 30 22:13:17.244260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:17.242749    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 30 22:13:17.244260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:17.242817    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc" podUID="dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 30 22:13:18.226330 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:18.226282    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 30 22:13:18.226527 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:18.226396    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc" podUID="dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 30 22:13:18.366522 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:13:18.366473    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:13:18.836590 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:18.836551    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:19.224271 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:19.222472    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:19.232619 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:19.232589    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:19.234007 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:19.233432    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:19.234283 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:19.234264    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:13:19.418329 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:19.418287    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:20.511269 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:20.507447    5586 scope.go:122] "RemoveContainer" containerID="6af4467d62c1b410e5e8d1ef4c7871535f4b7bca7016f6a546f0f2c596fcf3c7"
Oct 30 22:13:23.998765 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:23.998731    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:26.032715 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.032669    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"]
Oct 30 22:13:26.040957 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.040923    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.156429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.156387    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.156632 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.156437    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de56be49-5476-40e9-a737-0b12ccfdab60-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.156632 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.156597    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.156783 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.156683    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.156783 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.156716    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de56be49-5476-40e9-a737-0b12ccfdab60-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.156905 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.156807    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.156965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.156936    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de56be49-5476-40e9-a737-0b12ccfdab60-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.157024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.156986    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.157084 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.157060    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjsjq\" (UniqueName: \"kubernetes.io/projected/de56be49-5476-40e9-a737-0b12ccfdab60-kube-api-access-cjsjq\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.251132 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.251096    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"]
Oct 30 22:13:26.257982 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.257946    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de56be49-5476-40e9-a737-0b12ccfdab60-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.258163 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.258018    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.258163 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.258054    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjsjq\" (UniqueName: \"kubernetes.io/projected/de56be49-5476-40e9-a737-0b12ccfdab60-kube-api-access-cjsjq\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.258163 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.258106    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.258163 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.258155    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de56be49-5476-40e9-a737-0b12ccfdab60-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.258436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.258262    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.258436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.258331    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.258436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.258358    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de56be49-5476-40e9-a737-0b12ccfdab60-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.258436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.258404    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.258833 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.258797    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de56be49-5476-40e9-a737-0b12ccfdab60-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.259190 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.259164    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.259751 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.259214    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.259751 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.259274    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.259919 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.259465    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.261744 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.261718    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.261911 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.261890    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de56be49-5476-40e9-a737-0b12ccfdab60-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.315356 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.315282    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de56be49-5476-40e9-a737-0b12ccfdab60-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.315499 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.315376    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjsjq\" (UniqueName: \"kubernetes.io/projected/de56be49-5476-40e9-a737-0b12ccfdab60-kube-api-access-cjsjq\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.356453 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.356413    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:26.792434 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.792397    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"]
Oct 30 22:13:26.827684 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:13:26.827651    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde56be49_5476_40e9_a737_0b12ccfdab60.slice/crio-cf71747dae46bcef053321241488657f5164bb4a1a86967340977ea34f38ffe8 WatchSource:0}: Error finding container cf71747dae46bcef053321241488657f5164bb4a1a86967340977ea34f38ffe8: Status 404 returned error can't find the container with id cf71747dae46bcef053321241488657f5164bb4a1a86967340977ea34f38ffe8
Oct 30 22:13:26.835014 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.834980    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:26.835117 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.835054    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:26.835117 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:26.835085    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:27.818006 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:27.817965    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt" event={"ID":"de56be49-5476-40e9-a737-0b12ccfdab60","Type":"ContainerStarted","Data":"c73a7f805b41b55c5fa63bb3ec0f3ea5441153638520d494dcf40854e773140c"}
Oct 30 22:13:27.818572 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:27.818036    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt" event={"ID":"de56be49-5476-40e9-a737-0b12ccfdab60","Type":"ContainerStarted","Data":"cf71747dae46bcef053321241488657f5164bb4a1a86967340977ea34f38ffe8"}
Oct 30 22:13:27.998524 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:27.998460    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt" podStartSLOduration=2.9984470930000002 podStartE2EDuration="2.998447093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:27.947034078 +0000 UTC m=+2350.618938014" watchObservedRunningTime="2026-10-30 22:13:27.998447093 +0000 UTC m=+2350.670351028"
Oct 30 22:13:28.248938 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:28.248886    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:28.362742 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:28.358387    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:28.373258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:28.366358    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 22:13:28.373258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:28.366421    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt" podUID="de56be49-5476-40e9-a737-0b12ccfdab60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 22:13:29.210262 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:29.205450    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:29.357790 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:29.357750    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 22:13:29.357984 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:29.357805    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt" podUID="de56be49-5476-40e9-a737-0b12ccfdab60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 22:13:29.945594 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:29.945556    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:30.259320 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:30.259209    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:30.357843 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:30.357805    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 22:13:30.358036 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:30.357880    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt" podUID="de56be49-5476-40e9-a737-0b12ccfdab60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 22:13:30.487312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:30.487278    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:31.046452 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:31.044931    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:31.357528 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:31.357437    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 22:13:31.357528 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:31.357513    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt" podUID="de56be49-5476-40e9-a737-0b12ccfdab60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 22:13:32.357719 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:32.357678    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 22:13:32.358137 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:32.357741    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt" podUID="de56be49-5476-40e9-a737-0b12ccfdab60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 22:13:33.361839 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:33.361810    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:33.362267 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:33.362254    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:33.362985 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:33.362966    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:13:34.820749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:34.820708    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:36.263684 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:36.263641    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:36.480308 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:36.480261    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:36.643483 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:36.643383    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:37.182792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:37.182746    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:38.284104 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:38.284066    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:38.445084 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:38.445047    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:38.650803 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:38.650707    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:41.169290 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.169253    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"]
Oct 30 22:13:41.173790 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.173767    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.256357 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.256322    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"]
Oct 30 22:13:41.296649 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.296611    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v82hh\" (UniqueName: \"kubernetes.io/projected/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-kube-api-access-v82hh\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.296857 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.296656    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.296857 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.296799    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.296964 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.296867    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.296964 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.296920    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.297057 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.297006    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.297057 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.297039    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.297160 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.297064    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.297160 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.297093    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.398845 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.398805    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.399062 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.398916    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.399062 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.398974    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.399062 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.399044    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.399274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.399095    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.399274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.399129    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.399274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.399160    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.399274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.399200    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.399494 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.399343    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v82hh\" (UniqueName: \"kubernetes.io/projected/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-kube-api-access-v82hh\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.399494 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.399417    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-credential-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.399601 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.399489    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-workload-socket\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.399601 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.399530    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-workload-certs\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.399792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.399758    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-data\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.400037 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.400016    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istiod-ca-cert\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.401807 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.401784    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-podinfo\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.401909 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.401885    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-envoy\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.457121 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.457042    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-token\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.457298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.457170    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v82hh\" (UniqueName: \"kubernetes.io/projected/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-kube-api-access-v82hh\") pod \"e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") " pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.490226 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.490183    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:41.782463 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.782358    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:41.929252 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.929211    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"]
Oct 30 22:13:41.929592 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:13:41.929568    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c85fcdc_ee12_4b9b_8d38_ad82c0e398a5.slice/crio-c115fcf06943f5627e95f073301e23495892a54df9ebc0cf6c3c40d432dfbf52 WatchSource:0}: Error finding container c115fcf06943f5627e95f073301e23495892a54df9ebc0cf6c3c40d432dfbf52: Status 404 returned error can't find the container with id c115fcf06943f5627e95f073301e23495892a54df9ebc0cf6c3c40d432dfbf52
Oct 30 22:13:41.932569 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.932531    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:41.932696 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.932618    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:41.932696 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.932666    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:13:41.937751 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:41.937725    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:42.096535 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:42.096498    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:42.953549 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:42.953511    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk" event={"ID":"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5","Type":"ContainerStarted","Data":"2a58a13217a584025c66ea52f5f23d532d337f6723918157a0776c7e0ad24eb5"}
Oct 30 22:13:42.954075 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:42.953561    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk" event={"ID":"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5","Type":"ContainerStarted","Data":"c115fcf06943f5627e95f073301e23495892a54df9ebc0cf6c3c40d432dfbf52"}
Oct 30 22:13:43.171825 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:43.171756    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk" podStartSLOduration=2.171739873 podStartE2EDuration="2.171739873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:43.119767782 +0000 UTC m=+2365.791671721" watchObservedRunningTime="2026-10-30 22:13:43.171739873 +0000 UTC m=+2365.843643808"
Oct 30 22:13:43.490384 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:43.490349    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:43.492309 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:43.492282    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:13:43.492446 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:43.492335    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk" podUID="5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 22:13:44.278868 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:44.278829    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:44.448271 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:44.448202    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:44.490953 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:44.490920    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:13:44.491124 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:44.490970    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk" podUID="5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 22:13:44.646377 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:44.635551    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:45.190572 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:45.190533    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:45.491439 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:45.491349    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:13:45.491439 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:45.491406    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk" podUID="5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 22:13:46.491506 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:46.491462    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:13:46.492031 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:46.491533    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk" podUID="5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 22:13:47.264408 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:47.264363    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:47.491263 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:47.491194    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:13:47.491456 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:47.491319    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk" podUID="5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 22:13:48.363720 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:13:48.363689    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:13:48.505937 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:48.505902    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:48.506571 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:48.506541    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:48.507179 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:48.507158    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:13:50.270248 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:50.270205    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:51.690527 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:51.690485    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"]
Oct 30 22:13:51.691128 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:51.690755    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc" podUID="5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59" containerName="maas-api" containerID="cri-o://ce91548e25be14d637040bfef9e06dbb872fc7b35d5c19b9b647d71fe2fc8980" gracePeriod=30
Oct 30 22:13:51.789541 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:51.789503    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:52.000168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.000137    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:13:52.022617 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.022585    5586 generic.go:356] "Generic (PLEG): container finished" podID="5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59" containerID="ce91548e25be14d637040bfef9e06dbb872fc7b35d5c19b9b647d71fe2fc8980" exitCode=0
Oct 30 22:13:52.022827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.022639    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc" event={"ID":"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59","Type":"ContainerDied","Data":"ce91548e25be14d637040bfef9e06dbb872fc7b35d5c19b9b647d71fe2fc8980"}
Oct 30 22:13:52.022827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.022647    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"
Oct 30 22:13:52.022827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.022689    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc" event={"ID":"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59","Type":"ContainerDied","Data":"56e8d7101f2c3a267ae70b56837834885a8e121c32334f2ed24e5e8febbc66c5"}
Oct 30 22:13:52.022827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.022711    5586 scope.go:122] "RemoveContainer" containerID="ce91548e25be14d637040bfef9e06dbb872fc7b35d5c19b9b647d71fe2fc8980"
Oct 30 22:13:52.032691 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.032666    5586 scope.go:122] "RemoveContainer" containerID="ce91548e25be14d637040bfef9e06dbb872fc7b35d5c19b9b647d71fe2fc8980"
Oct 30 22:13:52.032982 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:13:52.032960    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce91548e25be14d637040bfef9e06dbb872fc7b35d5c19b9b647d71fe2fc8980\": container with ID starting with ce91548e25be14d637040bfef9e06dbb872fc7b35d5c19b9b647d71fe2fc8980 not found: ID does not exist" containerID="ce91548e25be14d637040bfef9e06dbb872fc7b35d5c19b9b647d71fe2fc8980"
Oct 30 22:13:52.033056 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.032994    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce91548e25be14d637040bfef9e06dbb872fc7b35d5c19b9b647d71fe2fc8980"} err="failed to get container status \"ce91548e25be14d637040bfef9e06dbb872fc7b35d5c19b9b647d71fe2fc8980\": rpc error: code = NotFound desc = could not find container \"ce91548e25be14d637040bfef9e06dbb872fc7b35d5c19b9b647d71fe2fc8980\": container with ID starting with ce91548e25be14d637040bfef9e06dbb872fc7b35d5c19b9b647d71fe2fc8980 not found: ID does not exist"
Oct 30 22:13:52.064563 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.064529    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-maas-api-tls\") pod \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") "
Oct 30 22:13:52.064795 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.064778    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-kube-api-access-qg52h\" (UniqueName: \"kubernetes.io/projected/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-kube-api-access-qg52h\") pod \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") "
Oct 30 22:13:52.064901 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.064885    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-metrics-certs\") pod \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") "
Oct 30 22:13:52.064985 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.064926    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-openshift-service-ca\") pod \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\" (UID: \"5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59\") "
Oct 30 22:13:52.065551 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.065518    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-openshift-service-ca" pod "5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59" (UID: "5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:13:52.067044 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.067013    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-metrics-certs" pod "5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59" (UID: "5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:13:52.067160 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.067022    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-kube-api-access-qg52h" pod "5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59" (UID: "5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59"). InnerVolumeSpecName "kube-api-access-qg52h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:13:52.067160 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.067091    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-maas-api-tls" pod "5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59" (UID: "5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:13:52.167026 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.166985    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qg52h\" (UniqueName: \"kubernetes.io/projected/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-kube-api-access-qg52h\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:52.167026 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.167022    5586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-metrics-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:52.167026 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.167032    5586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-openshift-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:52.167320 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.167041    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59-maas-api-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:13:52.447806 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.447770    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"]
Oct 30 22:13:52.452274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:52.452225    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6df3ba0e-68fdb6c769-4mslc"]
Oct 30 22:13:53.421551 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:53.421510    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:53.449511 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:53.449475    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59" path="/var/lib/kubelet/pods/5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59/volumes"
Oct 30 22:13:53.654545 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:53.654509    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:53.806338 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:53.806249    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:54.392500 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:54.392459    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:55.799823 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:55.799785    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:56.032041 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:56.032006    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"]
Oct 30 22:13:56.142349 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:56.142220    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"]
Oct 30 22:13:56.463405 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:56.463367    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:56.623314 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:56.623277    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:57.039143 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:57.039110    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:57.240547 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:57.240500    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:58.958426 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:58.958366    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:13:59.300010 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:59.299916    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"]
Oct 30 22:13:59.519606 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:13:59.519560    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:00.864767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:00.864719    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:01.032829 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.032786    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5" podUID="556f26e0-ea90-47c8-af47-19814cb09daa" containerName="payload-processing" containerID="cri-o://821269e7fce1eb7831ec49c04cb5ab03b10f342d7afa3c168daffdd01d69f509" gracePeriod=25
Oct 30 22:14:01.143962 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.143875    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf" podUID="daf26fab-89e6-4d8d-ad82-0919c933c32a" containerName="payload-pre-processing" containerID="cri-o://90083aeb46c6356fe6f1cafc2a553acc7c5842aade470a5642aaa7718ff29b56" gracePeriod=25
Oct 30 22:14:01.308291 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.308260    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"
Oct 30 22:14:01.420462 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.420433    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"
Oct 30 22:14:01.444534 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.444492    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/556f26e0-ea90-47c8-af47-19814cb09daa-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/556f26e0-ea90-47c8-af47-19814cb09daa-plugins-config-volume\") pod \"556f26e0-ea90-47c8-af47-19814cb09daa\" (UID: \"556f26e0-ea90-47c8-af47-19814cb09daa\") "
Oct 30 22:14:01.444738 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.444561    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/556f26e0-ea90-47c8-af47-19814cb09daa-kube-api-access-mnkm6\" (UniqueName: \"kubernetes.io/projected/556f26e0-ea90-47c8-af47-19814cb09daa-kube-api-access-mnkm6\") pod \"556f26e0-ea90-47c8-af47-19814cb09daa\" (UID: \"556f26e0-ea90-47c8-af47-19814cb09daa\") "
Oct 30 22:14:01.445001 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.444929    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/556f26e0-ea90-47c8-af47-19814cb09daa-plugins-config-volume" pod "556f26e0-ea90-47c8-af47-19814cb09daa" (UID: "556f26e0-ea90-47c8-af47-19814cb09daa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:01.445110 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.445085    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/556f26e0-ea90-47c8-af47-19814cb09daa-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:01.446840 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.446812    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/556f26e0-ea90-47c8-af47-19814cb09daa-kube-api-access-mnkm6" pod "556f26e0-ea90-47c8-af47-19814cb09daa" (UID: "556f26e0-ea90-47c8-af47-19814cb09daa"). InnerVolumeSpecName "kube-api-access-mnkm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:01.546459 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.546414    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/daf26fab-89e6-4d8d-ad82-0919c933c32a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daf26fab-89e6-4d8d-ad82-0919c933c32a-plugins-config-volume\") pod \"daf26fab-89e6-4d8d-ad82-0919c933c32a\" (UID: \"daf26fab-89e6-4d8d-ad82-0919c933c32a\") "
Oct 30 22:14:01.546679 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.546660    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daf26fab-89e6-4d8d-ad82-0919c933c32a-kube-api-access-697vx\" (UniqueName: \"kubernetes.io/projected/daf26fab-89e6-4d8d-ad82-0919c933c32a-kube-api-access-697vx\") pod \"daf26fab-89e6-4d8d-ad82-0919c933c32a\" (UID: \"daf26fab-89e6-4d8d-ad82-0919c933c32a\") "
Oct 30 22:14:01.546879 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.546856    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daf26fab-89e6-4d8d-ad82-0919c933c32a-plugins-config-volume" pod "daf26fab-89e6-4d8d-ad82-0919c933c32a" (UID: "daf26fab-89e6-4d8d-ad82-0919c933c32a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:01.547509 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.547289    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daf26fab-89e6-4d8d-ad82-0919c933c32a-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:01.547509 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.547315    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnkm6\" (UniqueName: \"kubernetes.io/projected/556f26e0-ea90-47c8-af47-19814cb09daa-kube-api-access-mnkm6\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:01.548916 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.548894    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daf26fab-89e6-4d8d-ad82-0919c933c32a-kube-api-access-697vx" pod "daf26fab-89e6-4d8d-ad82-0919c933c32a" (UID: "daf26fab-89e6-4d8d-ad82-0919c933c32a"). InnerVolumeSpecName "kube-api-access-697vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:01.648293 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:01.648227    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-697vx\" (UniqueName: \"kubernetes.io/projected/daf26fab-89e6-4d8d-ad82-0919c933c32a-kube-api-access-697vx\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:02.102352 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.102310    5586 generic.go:356] "Generic (PLEG): container finished" podID="daf26fab-89e6-4d8d-ad82-0919c933c32a" containerID="90083aeb46c6356fe6f1cafc2a553acc7c5842aade470a5642aaa7718ff29b56" exitCode=0
Oct 30 22:14:02.102830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.102366    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"
Oct 30 22:14:02.102830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.102407    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf" event={"ID":"daf26fab-89e6-4d8d-ad82-0919c933c32a","Type":"ContainerDied","Data":"90083aeb46c6356fe6f1cafc2a553acc7c5842aade470a5642aaa7718ff29b56"}
Oct 30 22:14:02.102830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.102467    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf" event={"ID":"daf26fab-89e6-4d8d-ad82-0919c933c32a","Type":"ContainerDied","Data":"5ea38fcd22e44af4eb76156d7a86de030939b668dcfd7c537ecbf499a3c14392"}
Oct 30 22:14:02.102830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.102498    5586 scope.go:122] "RemoveContainer" containerID="90083aeb46c6356fe6f1cafc2a553acc7c5842aade470a5642aaa7718ff29b56"
Oct 30 22:14:02.104066 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.104044    5586 generic.go:356] "Generic (PLEG): container finished" podID="556f26e0-ea90-47c8-af47-19814cb09daa" containerID="821269e7fce1eb7831ec49c04cb5ab03b10f342d7afa3c168daffdd01d69f509" exitCode=0
Oct 30 22:14:02.104183 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.104087    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5" event={"ID":"556f26e0-ea90-47c8-af47-19814cb09daa","Type":"ContainerDied","Data":"821269e7fce1eb7831ec49c04cb5ab03b10f342d7afa3c168daffdd01d69f509"}
Oct 30 22:14:02.104183 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.104104    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"
Oct 30 22:14:02.104183 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.104131    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5" event={"ID":"556f26e0-ea90-47c8-af47-19814cb09daa","Type":"ContainerDied","Data":"fc2f98057a38a35be85d7560184de44f786c6f5b67d150b089a3bbad3abb2b54"}
Oct 30 22:14:02.114092 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.114067    5586 scope.go:122] "RemoveContainer" containerID="90083aeb46c6356fe6f1cafc2a553acc7c5842aade470a5642aaa7718ff29b56"
Oct 30 22:14:02.114490 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:14:02.114468    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90083aeb46c6356fe6f1cafc2a553acc7c5842aade470a5642aaa7718ff29b56\": container with ID starting with 90083aeb46c6356fe6f1cafc2a553acc7c5842aade470a5642aaa7718ff29b56 not found: ID does not exist" containerID="90083aeb46c6356fe6f1cafc2a553acc7c5842aade470a5642aaa7718ff29b56"
Oct 30 22:14:02.114581 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.114503    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90083aeb46c6356fe6f1cafc2a553acc7c5842aade470a5642aaa7718ff29b56"} err="failed to get container status \"90083aeb46c6356fe6f1cafc2a553acc7c5842aade470a5642aaa7718ff29b56\": rpc error: code = NotFound desc = could not find container \"90083aeb46c6356fe6f1cafc2a553acc7c5842aade470a5642aaa7718ff29b56\": container with ID starting with 90083aeb46c6356fe6f1cafc2a553acc7c5842aade470a5642aaa7718ff29b56 not found: ID does not exist"
Oct 30 22:14:02.114581 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.114521    5586 scope.go:122] "RemoveContainer" containerID="821269e7fce1eb7831ec49c04cb5ab03b10f342d7afa3c168daffdd01d69f509"
Oct 30 22:14:02.130703 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.130671    5586 scope.go:122] "RemoveContainer" containerID="821269e7fce1eb7831ec49c04cb5ab03b10f342d7afa3c168daffdd01d69f509"
Oct 30 22:14:02.131037 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:14:02.131017    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"821269e7fce1eb7831ec49c04cb5ab03b10f342d7afa3c168daffdd01d69f509\": container with ID starting with 821269e7fce1eb7831ec49c04cb5ab03b10f342d7afa3c168daffdd01d69f509 not found: ID does not exist" containerID="821269e7fce1eb7831ec49c04cb5ab03b10f342d7afa3c168daffdd01d69f509"
Oct 30 22:14:02.131132 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.131045    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"821269e7fce1eb7831ec49c04cb5ab03b10f342d7afa3c168daffdd01d69f509"} err="failed to get container status \"821269e7fce1eb7831ec49c04cb5ab03b10f342d7afa3c168daffdd01d69f509\": rpc error: code = NotFound desc = could not find container \"821269e7fce1eb7831ec49c04cb5ab03b10f342d7afa3c168daffdd01d69f509\": container with ID starting with 821269e7fce1eb7831ec49c04cb5ab03b10f342d7afa3c168daffdd01d69f509 not found: ID does not exist"
Oct 30 22:14:02.283786 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.283751    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"]
Oct 30 22:14:02.297199 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.297167    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-c3e2d8-765f5bb5d9-z9zj5"]
Oct 30 22:14:02.413059 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.413032    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"]
Oct 30 22:14:02.418761 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:02.418732    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-c3e2d8-6c579b78bc-jbsnf"]
Oct 30 22:14:03.449143 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:03.449103    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="556f26e0-ea90-47c8-af47-19814cb09daa" path="/var/lib/kubelet/pods/556f26e0-ea90-47c8-af47-19814cb09daa/volumes"
Oct 30 22:14:03.449698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:03.449639    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="daf26fab-89e6-4d8d-ad82-0919c933c32a" path="/var/lib/kubelet/pods/daf26fab-89e6-4d8d-ad82-0919c933c32a/volumes"
Oct 30 22:14:03.998001 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:03.997965    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:06.746659 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:06.746603    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"]
Oct 30 22:14:06.849564 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:06.849516    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"]
Oct 30 22:14:07.122620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:07.122481    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:07.330816 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:07.330775    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:08.809528 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:08.809484    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:11.747210 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:11.747122    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp" podUID="9c7d9808-fd6c-480c-8b6a-1f3a47eb6541" containerName="payload-processing" containerID="cri-o://86c009b69912dff0354810d1be4e8fb3cc4651b30d9ef4dd2b0cf716f65873fc" gracePeriod=25
Oct 30 22:14:11.850464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:11.850425    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h" podUID="33ecbfc9-fecd-4f7a-b162-40bd00a040b9" containerName="payload-pre-processing" containerID="cri-o://753c02f3f27330cd76522e3a763ae6ba68e6abae22b343dfb145e4e48260965d" gracePeriod=25
Oct 30 22:14:13.074144 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.074105    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"]
Oct 30 22:14:13.074995 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.074972    5586 cpu_manager.go:414] "containerMap: removing container" podUID="556f26e0-ea90-47c8-af47-19814cb09daa" containerName="payload-processing"
Oct 30 22:14:13.074995 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.074994    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="556f26e0-ea90-47c8-af47-19814cb09daa" containerName="payload-processing"
Oct 30 22:14:13.075153 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.075018    5586 cpu_manager.go:414] "containerMap: removing container" podUID="daf26fab-89e6-4d8d-ad82-0919c933c32a" containerName="payload-pre-processing"
Oct 30 22:14:13.075153 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.075024    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="daf26fab-89e6-4d8d-ad82-0919c933c32a" containerName="payload-pre-processing"
Oct 30 22:14:13.075153 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.075051    5586 cpu_manager.go:414] "containerMap: removing container" podUID="5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59" containerName="maas-api"
Oct 30 22:14:13.075153 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.075056    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59" containerName="maas-api"
Oct 30 22:14:13.075153 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.075155    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c1c2bf2-e30f-41c9-ad29-0c1ab6960d59" containerName="maas-api"
Oct 30 22:14:13.075422 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.075170    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="556f26e0-ea90-47c8-af47-19814cb09daa" containerName="payload-processing"
Oct 30 22:14:13.075422 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.075181    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="daf26fab-89e6-4d8d-ad82-0919c933c32a" containerName="payload-pre-processing"
Oct 30 22:14:13.080505 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.080475    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"
Oct 30 22:14:13.116416 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.116375    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b4be694-b324-4fa7-a5fd-0703c94340e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-59f6a75f-mj7mf\" (UID: \"8b4be694-b324-4fa7-a5fd-0703c94340e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"
Oct 30 22:14:13.116603 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.116575    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b4be694-b324-4fa7-a5fd-0703c94340e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-59f6a75f-mj7mf\" (UID: \"8b4be694-b324-4fa7-a5fd-0703c94340e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"
Oct 30 22:14:13.116722 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.116701    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs55q\" (UniqueName: \"kubernetes.io/projected/8b4be694-b324-4fa7-a5fd-0703c94340e5-kube-api-access-zs55q\") pod \"maas-api-revoke-sub-59f6a75f-mj7mf\" (UID: \"8b4be694-b324-4fa7-a5fd-0703c94340e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"
Oct 30 22:14:13.138592 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.138554    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"]
Oct 30 22:14:13.183194 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.183160    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:14:13.183194 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.183160    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:14:13.200132 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.200093    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h" podUID="33ecbfc9-fecd-4f7a-b162-40bd00a040b9" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:14:13.200360 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.200100    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp" podUID="9c7d9808-fd6c-480c-8b6a-1f3a47eb6541" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:14:13.217680 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.217640    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b4be694-b324-4fa7-a5fd-0703c94340e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-59f6a75f-mj7mf\" (UID: \"8b4be694-b324-4fa7-a5fd-0703c94340e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"
Oct 30 22:14:13.217797 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.217718    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs55q\" (UniqueName: \"kubernetes.io/projected/8b4be694-b324-4fa7-a5fd-0703c94340e5-kube-api-access-zs55q\") pod \"maas-api-revoke-sub-59f6a75f-mj7mf\" (UID: \"8b4be694-b324-4fa7-a5fd-0703c94340e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"
Oct 30 22:14:13.217797 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.217772    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b4be694-b324-4fa7-a5fd-0703c94340e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-59f6a75f-mj7mf\" (UID: \"8b4be694-b324-4fa7-a5fd-0703c94340e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"
Oct 30 22:14:13.218355 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.218332    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b4be694-b324-4fa7-a5fd-0703c94340e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-59f6a75f-mj7mf\" (UID: \"8b4be694-b324-4fa7-a5fd-0703c94340e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"
Oct 30 22:14:13.220108 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.220084    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b4be694-b324-4fa7-a5fd-0703c94340e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-59f6a75f-mj7mf\" (UID: \"8b4be694-b324-4fa7-a5fd-0703c94340e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"
Oct 30 22:14:13.275497 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.275462    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs55q\" (UniqueName: \"kubernetes.io/projected/8b4be694-b324-4fa7-a5fd-0703c94340e5-kube-api-access-zs55q\") pod \"maas-api-revoke-sub-59f6a75f-mj7mf\" (UID: \"8b4be694-b324-4fa7-a5fd-0703c94340e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"
Oct 30 22:14:13.309164 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.309128    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"]
Oct 30 22:14:13.313482 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.313459    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"
Oct 30 22:14:13.371129 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.371030    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"]
Oct 30 22:14:13.398097 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.398061    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"
Oct 30 22:14:13.420716 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.420663    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slw75\" (UniqueName: \"kubernetes.io/projected/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-kube-api-access-slw75\") pod \"maas-api-revoke-sub-48b5b732-7w9gs\" (UID: \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"
Oct 30 22:14:13.420881 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.420803    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-48b5b732-7w9gs\" (UID: \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"
Oct 30 22:14:13.421001 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.420977    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-48b5b732-7w9gs\" (UID: \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"
Oct 30 22:14:13.522633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.522601    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slw75\" (UniqueName: \"kubernetes.io/projected/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-kube-api-access-slw75\") pod \"maas-api-revoke-sub-48b5b732-7w9gs\" (UID: \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"
Oct 30 22:14:13.522880 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.522664    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-48b5b732-7w9gs\" (UID: \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"
Oct 30 22:14:13.522880 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.522806    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-48b5b732-7w9gs\" (UID: \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"
Oct 30 22:14:13.523527 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.523494    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-48b5b732-7w9gs\" (UID: \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"
Oct 30 22:14:13.525512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.525479    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-48b5b732-7w9gs\" (UID: \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"
Oct 30 22:14:13.580356 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.580321    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slw75\" (UniqueName: \"kubernetes.io/projected/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-kube-api-access-slw75\") pod \"maas-api-revoke-sub-48b5b732-7w9gs\" (UID: \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"
Oct 30 22:14:13.631282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.631157    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"
Oct 30 22:14:13.786737 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:13.786708    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"]
Oct 30 22:14:13.788452 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:14:13.788422    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4be694_b324_4fa7_a5fd_0703c94340e5.slice/crio-1addf1e151dfe169c87555ee490c1c974dfb0941e08174591fed2249982ede70 WatchSource:0}: Error finding container 1addf1e151dfe169c87555ee490c1c974dfb0941e08174591fed2249982ede70: Status 404 returned error can't find the container with id 1addf1e151dfe169c87555ee490c1c974dfb0941e08174591fed2249982ede70
Oct 30 22:14:14.010716 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.010495    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"]
Oct 30 22:14:14.012345 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:14:14.012314    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e3476a_f9c3_42d6_bfe4_10348a474ba7.slice/crio-f06abf9ac9eac3d397e1418e52a8b664a5ef48738c1e67f61cd3ddfc9ab5ce9c WatchSource:0}: Error finding container f06abf9ac9eac3d397e1418e52a8b664a5ef48738c1e67f61cd3ddfc9ab5ce9c: Status 404 returned error can't find the container with id f06abf9ac9eac3d397e1418e52a8b664a5ef48738c1e67f61cd3ddfc9ab5ce9c
Oct 30 22:14:14.094630 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.094592    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:14.187189 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.187154    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf" event={"ID":"8b4be694-b324-4fa7-a5fd-0703c94340e5","Type":"ContainerStarted","Data":"1addf1e151dfe169c87555ee490c1c974dfb0941e08174591fed2249982ede70"}
Oct 30 22:14:14.188644 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.188613    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs" event={"ID":"f2e3476a-f9c3-42d6-bfe4-10348a474ba7","Type":"ContainerStarted","Data":"f06abf9ac9eac3d397e1418e52a8b664a5ef48738c1e67f61cd3ddfc9ab5ce9c"}
Oct 30 22:14:14.321778 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.321676    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt" podUID="92f71b18-38f6-4abe-ad38-afd53cbadeea" containerName="main" containerID="cri-o://1446bae01d56a7471668a82c71152ffcb90a1d3f3019b2122457650e88df998c" gracePeriod=45
Oct 30 22:14:14.601275 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.601222    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:14:14.739556 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.739523    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-home\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-home\") pod \"92f71b18-38f6-4abe-ad38-afd53cbadeea\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") "
Oct 30 22:14:14.739748 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.739592    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-dshm\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-dshm\") pod \"92f71b18-38f6-4abe-ad38-afd53cbadeea\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") "
Oct 30 22:14:14.739748 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.739653    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92f71b18-38f6-4abe-ad38-afd53cbadeea-tls-certs\" (UniqueName: \"kubernetes.io/secret/92f71b18-38f6-4abe-ad38-afd53cbadeea-tls-certs\") pod \"92f71b18-38f6-4abe-ad38-afd53cbadeea\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") "
Oct 30 22:14:14.739897 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.739809    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-model-cache\") pod \"92f71b18-38f6-4abe-ad38-afd53cbadeea\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") "
Oct 30 22:14:14.739897 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.739824    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-home" pod "92f71b18-38f6-4abe-ad38-afd53cbadeea" (UID: "92f71b18-38f6-4abe-ad38-afd53cbadeea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:14.739897 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.739881    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92f71b18-38f6-4abe-ad38-afd53cbadeea-kube-api-access-qjmsd\" (UniqueName: \"kubernetes.io/projected/92f71b18-38f6-4abe-ad38-afd53cbadeea-kube-api-access-qjmsd\") pod \"92f71b18-38f6-4abe-ad38-afd53cbadeea\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") "
Oct 30 22:14:14.740054 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.739911    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-tmp-dir\") pod \"92f71b18-38f6-4abe-ad38-afd53cbadeea\" (UID: \"92f71b18-38f6-4abe-ad38-afd53cbadeea\") "
Oct 30 22:14:14.740122 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.740107    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-model-cache" pod "92f71b18-38f6-4abe-ad38-afd53cbadeea" (UID: "92f71b18-38f6-4abe-ad38-afd53cbadeea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:14.740212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.740186    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-tmp-dir" pod "92f71b18-38f6-4abe-ad38-afd53cbadeea" (UID: "92f71b18-38f6-4abe-ad38-afd53cbadeea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:14.740463 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.740441    5586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-model-cache\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:14.740463 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.740465    5586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-tmp-dir\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:14.740617 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.740479    5586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-home\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:14.742003 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.741982    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-dshm" pod "92f71b18-38f6-4abe-ad38-afd53cbadeea" (UID: "92f71b18-38f6-4abe-ad38-afd53cbadeea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:14.742082 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.742044    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92f71b18-38f6-4abe-ad38-afd53cbadeea-tls-certs" pod "92f71b18-38f6-4abe-ad38-afd53cbadeea" (UID: "92f71b18-38f6-4abe-ad38-afd53cbadeea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:14:14.742151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.742076    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92f71b18-38f6-4abe-ad38-afd53cbadeea-kube-api-access-qjmsd" pod "92f71b18-38f6-4abe-ad38-afd53cbadeea" (UID: "92f71b18-38f6-4abe-ad38-afd53cbadeea"). InnerVolumeSpecName "kube-api-access-qjmsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:14.841910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.841822    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjmsd\" (UniqueName: \"kubernetes.io/projected/92f71b18-38f6-4abe-ad38-afd53cbadeea-kube-api-access-qjmsd\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:14.841910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.841858    5586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92f71b18-38f6-4abe-ad38-afd53cbadeea-dshm\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:14.841910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:14.841875    5586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92f71b18-38f6-4abe-ad38-afd53cbadeea-tls-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:15.198200 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:15.198166    5586 generic.go:356] "Generic (PLEG): container finished" podID="f2e3476a-f9c3-42d6-bfe4-10348a474ba7" containerID="3ab68cda2abf99e5701587a65cca7d83389636aea4268b39544aa8f0f150c40b" exitCode=0
Oct 30 22:14:15.198621 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:15.198282    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs" event={"ID":"f2e3476a-f9c3-42d6-bfe4-10348a474ba7","Type":"ContainerDied","Data":"3ab68cda2abf99e5701587a65cca7d83389636aea4268b39544aa8f0f150c40b"}
Oct 30 22:14:15.199723 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:15.199697    5586 generic.go:356] "Generic (PLEG): container finished" podID="92f71b18-38f6-4abe-ad38-afd53cbadeea" containerID="1446bae01d56a7471668a82c71152ffcb90a1d3f3019b2122457650e88df998c" exitCode=0
Oct 30 22:14:15.199832 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:15.199745    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"
Oct 30 22:14:15.199832 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:15.199775    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt" event={"ID":"92f71b18-38f6-4abe-ad38-afd53cbadeea","Type":"ContainerDied","Data":"1446bae01d56a7471668a82c71152ffcb90a1d3f3019b2122457650e88df998c"}
Oct 30 22:14:15.199832 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:15.199818    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt" event={"ID":"92f71b18-38f6-4abe-ad38-afd53cbadeea","Type":"ContainerDied","Data":"f11ce377cd7f2f42a7d032e849f452a4f25269c3a6718a778df143bcf160ae56"}
Oct 30 22:14:15.199993 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:15.199835    5586 scope.go:122] "RemoveContainer" containerID="1446bae01d56a7471668a82c71152ffcb90a1d3f3019b2122457650e88df998c"
Oct 30 22:14:15.213518 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:15.213481    5586 generic.go:356] "Generic (PLEG): container finished" podID="8b4be694-b324-4fa7-a5fd-0703c94340e5" containerID="e25500d3ff037bc91f26d97144297407911f0f8cecbb08cc97d260c6579db66b" exitCode=0
Oct 30 22:14:15.213656 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:15.213608    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf" event={"ID":"8b4be694-b324-4fa7-a5fd-0703c94340e5","Type":"ContainerDied","Data":"e25500d3ff037bc91f26d97144297407911f0f8cecbb08cc97d260c6579db66b"}
Oct 30 22:14:15.222127 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:15.222100    5586 scope.go:122] "RemoveContainer" containerID="1446bae01d56a7471668a82c71152ffcb90a1d3f3019b2122457650e88df998c"
Oct 30 22:14:15.222488 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:14:15.222462    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1446bae01d56a7471668a82c71152ffcb90a1d3f3019b2122457650e88df998c\": container with ID starting with 1446bae01d56a7471668a82c71152ffcb90a1d3f3019b2122457650e88df998c not found: ID does not exist" containerID="1446bae01d56a7471668a82c71152ffcb90a1d3f3019b2122457650e88df998c"
Oct 30 22:14:15.222553 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:15.222497    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1446bae01d56a7471668a82c71152ffcb90a1d3f3019b2122457650e88df998c"} err="failed to get container status \"1446bae01d56a7471668a82c71152ffcb90a1d3f3019b2122457650e88df998c\": rpc error: code = NotFound desc = could not find container \"1446bae01d56a7471668a82c71152ffcb90a1d3f3019b2122457650e88df998c\": container with ID starting with 1446bae01d56a7471668a82c71152ffcb90a1d3f3019b2122457650e88df998c not found: ID does not exist"
Oct 30 22:14:15.536900 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:15.536846    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"]
Oct 30 22:14:15.539917 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:15.539896    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt"]
Oct 30 22:14:15.574953 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:15.574906    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-6df3ba0e/e2e-lifecycle-model-6df3ba0e-kserve-57cbfc4c57-tpslt" podUID="92f71b18-38f6-4abe-ad38-afd53cbadeea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.131:8000/ready\": context deadline exceeded"
Oct 30 22:14:16.439497 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.439395    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"
Oct 30 22:14:16.442437 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.442411    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"
Oct 30 22:14:16.566331 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.566250    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-maas-api-xks-ca\") pod \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\" (UID: \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\") "
Oct 30 22:14:16.566331 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.566296    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-kube-api-access-slw75\" (UniqueName: \"kubernetes.io/projected/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-kube-api-access-slw75\") pod \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\" (UID: \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\") "
Oct 30 22:14:16.566693 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.566345    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-maas-api-openshift-ca\") pod \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\" (UID: \"f2e3476a-f9c3-42d6-bfe4-10348a474ba7\") "
Oct 30 22:14:16.566693 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.566364    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b4be694-b324-4fa7-a5fd-0703c94340e5-kube-api-access-zs55q\" (UniqueName: \"kubernetes.io/projected/8b4be694-b324-4fa7-a5fd-0703c94340e5-kube-api-access-zs55q\") pod \"8b4be694-b324-4fa7-a5fd-0703c94340e5\" (UID: \"8b4be694-b324-4fa7-a5fd-0703c94340e5\") "
Oct 30 22:14:16.566693 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.566415    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b4be694-b324-4fa7-a5fd-0703c94340e5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b4be694-b324-4fa7-a5fd-0703c94340e5-maas-api-xks-ca\") pod \"8b4be694-b324-4fa7-a5fd-0703c94340e5\" (UID: \"8b4be694-b324-4fa7-a5fd-0703c94340e5\") "
Oct 30 22:14:16.566693 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.566503    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b4be694-b324-4fa7-a5fd-0703c94340e5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b4be694-b324-4fa7-a5fd-0703c94340e5-maas-api-openshift-ca\") pod \"8b4be694-b324-4fa7-a5fd-0703c94340e5\" (UID: \"8b4be694-b324-4fa7-a5fd-0703c94340e5\") "
Oct 30 22:14:16.567058 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.567036    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-maas-api-openshift-ca" pod "f2e3476a-f9c3-42d6-bfe4-10348a474ba7" (UID: "f2e3476a-f9c3-42d6-bfe4-10348a474ba7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:16.567340 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.567314    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b4be694-b324-4fa7-a5fd-0703c94340e5-maas-api-openshift-ca" pod "8b4be694-b324-4fa7-a5fd-0703c94340e5" (UID: "8b4be694-b324-4fa7-a5fd-0703c94340e5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:16.569365 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.569338    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b4be694-b324-4fa7-a5fd-0703c94340e5-kube-api-access-zs55q" pod "8b4be694-b324-4fa7-a5fd-0703c94340e5" (UID: "8b4be694-b324-4fa7-a5fd-0703c94340e5"). InnerVolumeSpecName "kube-api-access-zs55q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:16.570599 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.570573    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-kube-api-access-slw75" pod "f2e3476a-f9c3-42d6-bfe4-10348a474ba7" (UID: "f2e3476a-f9c3-42d6-bfe4-10348a474ba7"). InnerVolumeSpecName "kube-api-access-slw75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:16.570702 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.570610    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-maas-api-xks-ca" pod "f2e3476a-f9c3-42d6-bfe4-10348a474ba7" (UID: "f2e3476a-f9c3-42d6-bfe4-10348a474ba7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:14:16.570702 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.570633    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b4be694-b324-4fa7-a5fd-0703c94340e5-maas-api-xks-ca" pod "8b4be694-b324-4fa7-a5fd-0703c94340e5" (UID: "8b4be694-b324-4fa7-a5fd-0703c94340e5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:14:16.668724 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.668693    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8b4be694-b324-4fa7-a5fd-0703c94340e5-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.668724 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.668721    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8b4be694-b324-4fa7-a5fd-0703c94340e5-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.668724 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.668732    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.668970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.668740    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slw75\" (UniqueName: \"kubernetes.io/projected/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-kube-api-access-slw75\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.668970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.668751    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f2e3476a-f9c3-42d6-bfe4-10348a474ba7-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.668970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:16.668759    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs55q\" (UniqueName: \"kubernetes.io/projected/8b4be694-b324-4fa7-a5fd-0703c94340e5-kube-api-access-zs55q\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:17.235124 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:17.235091    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"
Oct 30 22:14:17.235124 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:17.235104    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf" event={"ID":"8b4be694-b324-4fa7-a5fd-0703c94340e5","Type":"ContainerDied","Data":"1addf1e151dfe169c87555ee490c1c974dfb0941e08174591fed2249982ede70"}
Oct 30 22:14:17.235395 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:17.235160    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1addf1e151dfe169c87555ee490c1c974dfb0941e08174591fed2249982ede70"
Oct 30 22:14:17.236711 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:17.236691    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"
Oct 30 22:14:17.236877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:17.236731    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs" event={"ID":"f2e3476a-f9c3-42d6-bfe4-10348a474ba7","Type":"ContainerDied","Data":"f06abf9ac9eac3d397e1418e52a8b664a5ef48738c1e67f61cd3ddfc9ab5ce9c"}
Oct 30 22:14:17.236877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:17.236780    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f06abf9ac9eac3d397e1418e52a8b664a5ef48738c1e67f61cd3ddfc9ab5ce9c"
Oct 30 22:14:17.449464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:17.449432    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92f71b18-38f6-4abe-ad38-afd53cbadeea" path="/var/lib/kubelet/pods/92f71b18-38f6-4abe-ad38-afd53cbadeea/volumes"
Oct 30 22:14:17.977441 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:17.977413    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:14:17.982665 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:17.982637    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:14:17.983212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:17.983186    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:14:17.988538 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:17.988516    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:14:18.360897 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:14:18.360806    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:14:18.886361 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:18.886323    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:23.200151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:23.200114    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h" podUID="33ecbfc9-fecd-4f7a-b162-40bd00a040b9" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:14:23.200616 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:23.200125    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp" podUID="9c7d9808-fd6c-480c-8b6a-1f3a47eb6541" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:14:23.769708 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:23.769665    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:26.308312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:26.308271    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:26.501249 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:26.501192    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:27.112310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:27.112271    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:27.312926 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:27.312887    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:29.001773 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:29.001733    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:29.574998 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:29.574950    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:30.169637 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:30.169598    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:31.466226 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:31.466186    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:32.098750 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.098431    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"
Oct 30 22:14:32.210725 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.210695    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"
Oct 30 22:14:32.214996 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.214963    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541-kube-api-access-xlrm8\" (UniqueName: \"kubernetes.io/projected/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541-kube-api-access-xlrm8\") pod \"9c7d9808-fd6c-480c-8b6a-1f3a47eb6541\" (UID: \"9c7d9808-fd6c-480c-8b6a-1f3a47eb6541\") "
Oct 30 22:14:32.215198 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.215049    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541-plugins-config-volume\") pod \"9c7d9808-fd6c-480c-8b6a-1f3a47eb6541\" (UID: \"9c7d9808-fd6c-480c-8b6a-1f3a47eb6541\") "
Oct 30 22:14:32.215726 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.215691    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541-plugins-config-volume" pod "9c7d9808-fd6c-480c-8b6a-1f3a47eb6541" (UID: "9c7d9808-fd6c-480c-8b6a-1f3a47eb6541"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:32.217859 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.217829    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:32.218064 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.218042    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541-kube-api-access-xlrm8" pod "9c7d9808-fd6c-480c-8b6a-1f3a47eb6541" (UID: "9c7d9808-fd6c-480c-8b6a-1f3a47eb6541"). InnerVolumeSpecName "kube-api-access-xlrm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:32.319299 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.319174    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33ecbfc9-fecd-4f7a-b162-40bd00a040b9-kube-api-access-m7gb9\" (UniqueName: \"kubernetes.io/projected/33ecbfc9-fecd-4f7a-b162-40bd00a040b9-kube-api-access-m7gb9\") pod \"33ecbfc9-fecd-4f7a-b162-40bd00a040b9\" (UID: \"33ecbfc9-fecd-4f7a-b162-40bd00a040b9\") "
Oct 30 22:14:32.319299 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.319263    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33ecbfc9-fecd-4f7a-b162-40bd00a040b9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33ecbfc9-fecd-4f7a-b162-40bd00a040b9-plugins-config-volume\") pod \"33ecbfc9-fecd-4f7a-b162-40bd00a040b9\" (UID: \"33ecbfc9-fecd-4f7a-b162-40bd00a040b9\") "
Oct 30 22:14:32.319621 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.319608    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlrm8\" (UniqueName: \"kubernetes.io/projected/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541-kube-api-access-xlrm8\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:32.319685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.319604    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33ecbfc9-fecd-4f7a-b162-40bd00a040b9-plugins-config-volume" pod "33ecbfc9-fecd-4f7a-b162-40bd00a040b9" (UID: "33ecbfc9-fecd-4f7a-b162-40bd00a040b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:32.353416 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.351933    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33ecbfc9-fecd-4f7a-b162-40bd00a040b9-kube-api-access-m7gb9" pod "33ecbfc9-fecd-4f7a-b162-40bd00a040b9" (UID: "33ecbfc9-fecd-4f7a-b162-40bd00a040b9"). InnerVolumeSpecName "kube-api-access-m7gb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:32.368398 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.368361    5586 generic.go:356] "Generic (PLEG): container finished" podID="9c7d9808-fd6c-480c-8b6a-1f3a47eb6541" containerID="86c009b69912dff0354810d1be4e8fb3cc4651b30d9ef4dd2b0cf716f65873fc" exitCode=0
Oct 30 22:14:32.368398 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.368415    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"
Oct 30 22:14:32.368687 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.368454    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp" event={"ID":"9c7d9808-fd6c-480c-8b6a-1f3a47eb6541","Type":"ContainerDied","Data":"86c009b69912dff0354810d1be4e8fb3cc4651b30d9ef4dd2b0cf716f65873fc"}
Oct 30 22:14:32.368687 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.368519    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp" event={"ID":"9c7d9808-fd6c-480c-8b6a-1f3a47eb6541","Type":"ContainerDied","Data":"9d70a304e59596ee357121bd0742174e2359198130572018fc9a33ca563ce454"}
Oct 30 22:14:32.368687 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.368556    5586 scope.go:122] "RemoveContainer" containerID="86c009b69912dff0354810d1be4e8fb3cc4651b30d9ef4dd2b0cf716f65873fc"
Oct 30 22:14:32.370280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.370226    5586 generic.go:356] "Generic (PLEG): container finished" podID="33ecbfc9-fecd-4f7a-b162-40bd00a040b9" containerID="753c02f3f27330cd76522e3a763ae6ba68e6abae22b343dfb145e4e48260965d" exitCode=0
Oct 30 22:14:32.370417 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.370369    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h" event={"ID":"33ecbfc9-fecd-4f7a-b162-40bd00a040b9","Type":"ContainerDied","Data":"753c02f3f27330cd76522e3a763ae6ba68e6abae22b343dfb145e4e48260965d"}
Oct 30 22:14:32.370486 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.370414    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h" event={"ID":"33ecbfc9-fecd-4f7a-b162-40bd00a040b9","Type":"ContainerDied","Data":"4421b84d7dcd73739e62caebf3573adf5437a77524341586234b946077b93743"}
Oct 30 22:14:32.370544 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.370494    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"
Oct 30 22:14:32.387725 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.387695    5586 scope.go:122] "RemoveContainer" containerID="86c009b69912dff0354810d1be4e8fb3cc4651b30d9ef4dd2b0cf716f65873fc"
Oct 30 22:14:32.388105 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:14:32.388079    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86c009b69912dff0354810d1be4e8fb3cc4651b30d9ef4dd2b0cf716f65873fc\": container with ID starting with 86c009b69912dff0354810d1be4e8fb3cc4651b30d9ef4dd2b0cf716f65873fc not found: ID does not exist" containerID="86c009b69912dff0354810d1be4e8fb3cc4651b30d9ef4dd2b0cf716f65873fc"
Oct 30 22:14:32.388270 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.388111    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86c009b69912dff0354810d1be4e8fb3cc4651b30d9ef4dd2b0cf716f65873fc"} err="failed to get container status \"86c009b69912dff0354810d1be4e8fb3cc4651b30d9ef4dd2b0cf716f65873fc\": rpc error: code = NotFound desc = could not find container \"86c009b69912dff0354810d1be4e8fb3cc4651b30d9ef4dd2b0cf716f65873fc\": container with ID starting with 86c009b69912dff0354810d1be4e8fb3cc4651b30d9ef4dd2b0cf716f65873fc not found: ID does not exist"
Oct 30 22:14:32.388270 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.388138    5586 scope.go:122] "RemoveContainer" containerID="753c02f3f27330cd76522e3a763ae6ba68e6abae22b343dfb145e4e48260965d"
Oct 30 22:14:32.398937 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.398912    5586 scope.go:122] "RemoveContainer" containerID="753c02f3f27330cd76522e3a763ae6ba68e6abae22b343dfb145e4e48260965d"
Oct 30 22:14:32.399311 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:14:32.399291    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"753c02f3f27330cd76522e3a763ae6ba68e6abae22b343dfb145e4e48260965d\": container with ID starting with 753c02f3f27330cd76522e3a763ae6ba68e6abae22b343dfb145e4e48260965d not found: ID does not exist" containerID="753c02f3f27330cd76522e3a763ae6ba68e6abae22b343dfb145e4e48260965d"
Oct 30 22:14:32.399399 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.399320    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"753c02f3f27330cd76522e3a763ae6ba68e6abae22b343dfb145e4e48260965d"} err="failed to get container status \"753c02f3f27330cd76522e3a763ae6ba68e6abae22b343dfb145e4e48260965d\": rpc error: code = NotFound desc = could not find container \"753c02f3f27330cd76522e3a763ae6ba68e6abae22b343dfb145e4e48260965d\": container with ID starting with 753c02f3f27330cd76522e3a763ae6ba68e6abae22b343dfb145e4e48260965d not found: ID does not exist"
Oct 30 22:14:32.445260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.421442    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33ecbfc9-fecd-4f7a-b162-40bd00a040b9-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:32.445260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.421477    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7gb9\" (UniqueName: \"kubernetes.io/projected/33ecbfc9-fecd-4f7a-b162-40bd00a040b9-kube-api-access-m7gb9\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:32.571075 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.570981    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"]
Oct 30 22:14:32.573843 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.573818    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-557708-7f89484b55-x966h"]
Oct 30 22:14:32.690345 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.690297    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"]
Oct 30 22:14:32.692953 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.692919    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:32.694661 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:32.694638    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-557708-57fcc65d7f-bdcrp"]
Oct 30 22:14:33.458665 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:33.458618    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33ecbfc9-fecd-4f7a-b162-40bd00a040b9" path="/var/lib/kubelet/pods/33ecbfc9-fecd-4f7a-b162-40bd00a040b9/volumes"
Oct 30 22:14:33.459156 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:33.459136    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9c7d9808-fd6c-480c-8b6a-1f3a47eb6541" path="/var/lib/kubelet/pods/9c7d9808-fd6c-480c-8b6a-1f3a47eb6541/volumes"
Oct 30 22:14:34.431319 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:34.431272    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:35.854269 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:35.851517    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:37.178050 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:37.178005    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:37.353837 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:37.353802    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:37.691107 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:37.691073    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:38.424891 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:38.424853    5586 generic.go:356] "Generic (PLEG): container finished" podID="26432f05-beba-4093-a139-217dac85db7e" containerID="82b93ee8f4e0657f07b778d752df9415be6df278addfdfb9a327430e339434cb" exitCode=0
Oct 30 22:14:38.425423 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:38.424895    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs" event={"ID":"26432f05-beba-4093-a139-217dac85db7e","Type":"ContainerDied","Data":"82b93ee8f4e0657f07b778d752df9415be6df278addfdfb9a327430e339434cb"}
Oct 30 22:14:38.425535 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:38.425514    5586 scope.go:122] "RemoveContainer" containerID="82b93ee8f4e0657f07b778d752df9415be6df278addfdfb9a327430e339434cb"
Oct 30 22:14:38.427061 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:38.427024    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:14:38.427162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:38.427103    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:14:38.427162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:38.427138    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:14:38.848373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:38.848276    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:39.008201 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:39.008160    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:39.267260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:39.244121    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:39.456765 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:39.456729    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs" event={"ID":"26432f05-beba-4093-a139-217dac85db7e","Type":"ContainerStarted","Data":"7b474842685af055af4df70c9fc33183c360aa6c54b16b78f0fad2720e923420"}
Oct 30 22:14:39.942111 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:39.937003    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:40.009770 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:40.009734    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:14:40.011422 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:40.011388    5586 patch_prober.go:29] interesting pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 22:14:40.011501 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:40.011440    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs" podUID="26432f05-beba-4093-a139-217dac85db7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 22:14:40.376995 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:40.376899    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:41.010099 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:41.010060    5586 patch_prober.go:29] interesting pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 22:14:41.010568 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:41.010125    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs" podUID="26432f05-beba-4093-a139-217dac85db7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 22:14:41.343471 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:41.343370    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:14:42.010373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:42.010335    5586 patch_prober.go:29] interesting pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 22:14:42.010832 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:42.010393    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs" podUID="26432f05-beba-4093-a139-217dac85db7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 22:14:42.985681 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:42.985641    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:43.010191 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:43.010157    5586 patch_prober.go:29] interesting pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 22:14:43.010370 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:43.010214    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs" podUID="26432f05-beba-4093-a139-217dac85db7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 22:14:43.468787 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:43.468755    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"]
Oct 30 22:14:43.578293 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:43.577074    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"]
Oct 30 22:14:43.870594 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:43.870498    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:44.010434 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:44.010396    5586 patch_prober.go:29] interesting pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 22:14:44.010434 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:44.010465    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs" podUID="26432f05-beba-4093-a139-217dac85db7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 22:14:44.053094 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:44.050017    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:45.033282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:45.033245    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:14:45.035616 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:45.035590    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:14:45.091592 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:45.091541    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:46.418559 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:46.418513    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:48.362283 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:14:48.362228    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:14:48.470068 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:48.470021    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh" podUID="a0622aae-1702-4cb1-80b6-8740c5261667" containerName="payload-processing" containerID="cri-o://18b92ea91d1fdaf4539ad1e91021dde2b92320eb37de3472069efa1eded9f3dc" gracePeriod=25
Oct 30 22:14:48.577852 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:48.577791    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx" podUID="2d4ab435-292c-4532-a4de-412d74acc5ae" containerName="payload-pre-processing" containerID="cri-o://9c6fdba8a0f37b44706ee7aad0f8d908c8cfc31c2543411f0071c952c3918a9f" gracePeriod=25
Oct 30 22:14:50.183886 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:50.183851    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:51.185518 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:51.185479    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx" podUID="2d4ab435-292c-4532-a4de-412d74acc5ae" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:14:51.185956 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:51.185483    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh" podUID="a0622aae-1702-4cb1-80b6-8740c5261667" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:14:54.983922 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:54.983869    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:14:59.251734 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:14:59.251635    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:00.203287 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.203251    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"]
Oct 30 22:15:00.204103 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204085    5586 cpu_manager.go:414] "containerMap: removing container" podUID="8b4be694-b324-4fa7-a5fd-0703c94340e5" containerName="revoke-keys"
Oct 30 22:15:00.204157 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204108    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b4be694-b324-4fa7-a5fd-0703c94340e5" containerName="revoke-keys"
Oct 30 22:15:00.204157 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204141    5586 cpu_manager.go:414] "containerMap: removing container" podUID="92f71b18-38f6-4abe-ad38-afd53cbadeea" containerName="main"
Oct 30 22:15:00.204157 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204149    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92f71b18-38f6-4abe-ad38-afd53cbadeea" containerName="main"
Oct 30 22:15:00.204267 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204164    5586 cpu_manager.go:414] "containerMap: removing container" podUID="33ecbfc9-fecd-4f7a-b162-40bd00a040b9" containerName="payload-pre-processing"
Oct 30 22:15:00.204267 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204173    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33ecbfc9-fecd-4f7a-b162-40bd00a040b9" containerName="payload-pre-processing"
Oct 30 22:15:00.204267 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204185    5586 cpu_manager.go:414] "containerMap: removing container" podUID="9c7d9808-fd6c-480c-8b6a-1f3a47eb6541" containerName="payload-processing"
Oct 30 22:15:00.204267 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204192    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c7d9808-fd6c-480c-8b6a-1f3a47eb6541" containerName="payload-processing"
Oct 30 22:15:00.204267 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204211    5586 cpu_manager.go:414] "containerMap: removing container" podUID="f2e3476a-f9c3-42d6-bfe4-10348a474ba7" containerName="revoke-keys"
Oct 30 22:15:00.204267 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204219    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2e3476a-f9c3-42d6-bfe4-10348a474ba7" containerName="revoke-keys"
Oct 30 22:15:00.204449 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204404    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b4be694-b324-4fa7-a5fd-0703c94340e5" containerName="revoke-keys"
Oct 30 22:15:00.204449 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204419    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="92f71b18-38f6-4abe-ad38-afd53cbadeea" containerName="main"
Oct 30 22:15:00.204449 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204429    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="33ecbfc9-fecd-4f7a-b162-40bd00a040b9" containerName="payload-pre-processing"
Oct 30 22:15:00.204449 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204437    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c7d9808-fd6c-480c-8b6a-1f3a47eb6541" containerName="payload-processing"
Oct 30 22:15:00.204571 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.204449    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2e3476a-f9c3-42d6-bfe4-10348a474ba7" containerName="revoke-keys"
Oct 30 22:15:00.208639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.208617    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"]
Oct 30 22:15:00.208772 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.208731    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"
Oct 30 22:15:00.212775 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.212754    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"
Oct 30 22:15:00.238431 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.238402    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"]
Oct 30 22:15:00.242996 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.242972    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"
Oct 30 22:15:00.253308 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.253283    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"]
Oct 30 22:15:00.257690 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.257671    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"
Oct 30 22:15:00.311632 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.308332    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"]
Oct 30 22:15:00.312542 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.312512    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"]
Oct 30 22:15:00.315521 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.315489    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad8b80f-746c-4450-a732-3551846186c7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4\" (UID: \"7ad8b80f-746c-4450-a732-3551846186c7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"
Oct 30 22:15:00.315874 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.315846    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2xqt\" (UniqueName: \"kubernetes.io/projected/003da65f-5b3a-4735-bf4d-8f064ad48d96-kube-api-access-v2xqt\") pod \"maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8\" (UID: \"003da65f-5b3a-4735-bf4d-8f064ad48d96\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"
Oct 30 22:15:00.316301 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.316264    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drrtx\" (UniqueName: \"kubernetes.io/projected/7ad8b80f-746c-4450-a732-3551846186c7-kube-api-access-drrtx\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4\" (UID: \"7ad8b80f-746c-4450-a732-3551846186c7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"
Oct 30 22:15:00.316556 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.316536    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdl7z\" (UniqueName: \"kubernetes.io/projected/07444e4a-f484-4ff4-bffd-9c3f329fca00-kube-api-access-rdl7z\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh\" (UID: \"07444e4a-f484-4ff4-bffd-9c3f329fca00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"
Oct 30 22:15:00.316725 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.316703    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/003da65f-5b3a-4735-bf4d-8f064ad48d96-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8\" (UID: \"003da65f-5b3a-4735-bf4d-8f064ad48d96\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"
Oct 30 22:15:00.316827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.316801    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e70ec99-2141-4616-b598-84675558b246-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w\" (UID: \"6e70ec99-2141-4616-b598-84675558b246\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"
Oct 30 22:15:00.316923 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.316848    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07444e4a-f484-4ff4-bffd-9c3f329fca00-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh\" (UID: \"07444e4a-f484-4ff4-bffd-9c3f329fca00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"
Oct 30 22:15:00.316923 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.316885    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr4js\" (UniqueName: \"kubernetes.io/projected/6e70ec99-2141-4616-b598-84675558b246-kube-api-access-zr4js\") pod \"maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w\" (UID: \"6e70ec99-2141-4616-b598-84675558b246\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"
Oct 30 22:15:00.317169 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.317142    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"]
Oct 30 22:15:00.354414 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.354374    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:15:00.354660 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.354632    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"]
Oct 30 22:15:00.418450 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.418411    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drrtx\" (UniqueName: \"kubernetes.io/projected/7ad8b80f-746c-4450-a732-3551846186c7-kube-api-access-drrtx\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4\" (UID: \"7ad8b80f-746c-4450-a732-3551846186c7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"
Oct 30 22:15:00.418663 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.418507    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdl7z\" (UniqueName: \"kubernetes.io/projected/07444e4a-f484-4ff4-bffd-9c3f329fca00-kube-api-access-rdl7z\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh\" (UID: \"07444e4a-f484-4ff4-bffd-9c3f329fca00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"
Oct 30 22:15:00.418663 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.418556    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/003da65f-5b3a-4735-bf4d-8f064ad48d96-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8\" (UID: \"003da65f-5b3a-4735-bf4d-8f064ad48d96\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"
Oct 30 22:15:00.418663 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.418628    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e70ec99-2141-4616-b598-84675558b246-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w\" (UID: \"6e70ec99-2141-4616-b598-84675558b246\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"
Oct 30 22:15:00.418862 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.418679    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07444e4a-f484-4ff4-bffd-9c3f329fca00-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh\" (UID: \"07444e4a-f484-4ff4-bffd-9c3f329fca00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"
Oct 30 22:15:00.418862 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.418712    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr4js\" (UniqueName: \"kubernetes.io/projected/6e70ec99-2141-4616-b598-84675558b246-kube-api-access-zr4js\") pod \"maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w\" (UID: \"6e70ec99-2141-4616-b598-84675558b246\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"
Oct 30 22:15:00.418862 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.418786    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad8b80f-746c-4450-a732-3551846186c7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4\" (UID: \"7ad8b80f-746c-4450-a732-3551846186c7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"
Oct 30 22:15:00.418862 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.418843    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2xqt\" (UniqueName: \"kubernetes.io/projected/003da65f-5b3a-4735-bf4d-8f064ad48d96-kube-api-access-v2xqt\") pod \"maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8\" (UID: \"003da65f-5b3a-4735-bf4d-8f064ad48d96\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"
Oct 30 22:15:00.419478 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.419456    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/003da65f-5b3a-4735-bf4d-8f064ad48d96-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8\" (UID: \"003da65f-5b3a-4735-bf4d-8f064ad48d96\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"
Oct 30 22:15:00.419541 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.419480    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e70ec99-2141-4616-b598-84675558b246-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w\" (UID: \"6e70ec99-2141-4616-b598-84675558b246\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"
Oct 30 22:15:00.419589 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.419560    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07444e4a-f484-4ff4-bffd-9c3f329fca00-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh\" (UID: \"07444e4a-f484-4ff4-bffd-9c3f329fca00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"
Oct 30 22:15:00.419627 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.419560    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad8b80f-746c-4450-a732-3551846186c7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4\" (UID: \"7ad8b80f-746c-4450-a732-3551846186c7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"
Oct 30 22:15:00.467465 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.467369    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:00.480625 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.480587    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drrtx\" (UniqueName: \"kubernetes.io/projected/7ad8b80f-746c-4450-a732-3551846186c7-kube-api-access-drrtx\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4\" (UID: \"7ad8b80f-746c-4450-a732-3551846186c7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"
Oct 30 22:15:00.480777 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.480652    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdl7z\" (UniqueName: \"kubernetes.io/projected/07444e4a-f484-4ff4-bffd-9c3f329fca00-kube-api-access-rdl7z\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh\" (UID: \"07444e4a-f484-4ff4-bffd-9c3f329fca00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"
Oct 30 22:15:00.480777 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.480681    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr4js\" (UniqueName: \"kubernetes.io/projected/6e70ec99-2141-4616-b598-84675558b246-kube-api-access-zr4js\") pod \"maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w\" (UID: \"6e70ec99-2141-4616-b598-84675558b246\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"
Oct 30 22:15:00.481936 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.481915    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2xqt\" (UniqueName: \"kubernetes.io/projected/003da65f-5b3a-4735-bf4d-8f064ad48d96-kube-api-access-v2xqt\") pod \"maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8\" (UID: \"003da65f-5b3a-4735-bf4d-8f064ad48d96\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"
Oct 30 22:15:00.525651 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.525616    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"
Oct 30 22:15:00.529920 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.529886    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"
Oct 30 22:15:00.557269 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.557190    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"
Oct 30 22:15:00.576492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.576452    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"
Oct 30 22:15:00.871595 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.871556    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:00.985084 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:00.985056    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"]
Oct 30 22:15:00.986957 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:15:00.986929    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e70ec99_2141_4616_b598_84675558b246.slice/crio-590be24803e328aa46b17bf2da3e33a645613faee0f88701d0413d07e816fc9a WatchSource:0}: Error finding container 590be24803e328aa46b17bf2da3e33a645613faee0f88701d0413d07e816fc9a: Status 404 returned error can't find the container with id 590be24803e328aa46b17bf2da3e33a645613faee0f88701d0413d07e816fc9a
Oct 30 22:15:01.034068 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.034033    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"]
Oct 30 22:15:01.038514 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:15:01.038477    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07444e4a_f484_4ff4_bffd_9c3f329fca00.slice/crio-e43282c40a9d29471eec3afbadc937fc1653e497a839100e8ee373d363c6f288 WatchSource:0}: Error finding container e43282c40a9d29471eec3afbadc937fc1653e497a839100e8ee373d363c6f288: Status 404 returned error can't find the container with id e43282c40a9d29471eec3afbadc937fc1653e497a839100e8ee373d363c6f288
Oct 30 22:15:01.044428 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.041387    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"]
Oct 30 22:15:01.046305 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.045956    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"]
Oct 30 22:15:01.056590 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:15:01.056565    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod003da65f_5b3a_4735_bf4d_8f064ad48d96.slice/crio-2ac0d2e9cbbd62a7700180b99155fb9c136b209c067d89818578fdad76be37d4 WatchSource:0}: Error finding container 2ac0d2e9cbbd62a7700180b99155fb9c136b209c067d89818578fdad76be37d4: Status 404 returned error can't find the container with id 2ac0d2e9cbbd62a7700180b99155fb9c136b209c067d89818578fdad76be37d4
Oct 30 22:15:01.185732 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.185693    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx" podUID="2d4ab435-292c-4532-a4de-412d74acc5ae" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:15:01.185922 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.185740    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh" podUID="a0622aae-1702-4cb1-80b6-8740c5261667" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:15:01.193272 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.193226    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:01.624665 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.624556    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4" event={"ID":"7ad8b80f-746c-4450-a732-3551846186c7","Type":"ContainerStarted","Data":"1299760c09be9ac803dab8ef53096316c15dd4304621dc29ac67e1814c45b7c0"}
Oct 30 22:15:01.624665 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.624639    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4" event={"ID":"7ad8b80f-746c-4450-a732-3551846186c7","Type":"ContainerStarted","Data":"edc5c927ae02be70e50ccc07aa47fd95f9b39e459961c8fb277229f952d3e739"}
Oct 30 22:15:01.627791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.627754    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh" event={"ID":"07444e4a-f484-4ff4-bffd-9c3f329fca00","Type":"ContainerStarted","Data":"79120f91a1b2ab74ba247b9403c37415c168bcd22a0f45bc26c60d01142f6b6f"}
Oct 30 22:15:01.627976 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.627814    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh" event={"ID":"07444e4a-f484-4ff4-bffd-9c3f329fca00","Type":"ContainerStarted","Data":"e43282c40a9d29471eec3afbadc937fc1653e497a839100e8ee373d363c6f288"}
Oct 30 22:15:01.629828 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.629796    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w" event={"ID":"6e70ec99-2141-4616-b598-84675558b246","Type":"ContainerStarted","Data":"45ecf3b7243e69002100205ff2c64a32d32ae39cdbf9eccd6241ec605e35b547"}
Oct 30 22:15:01.629982 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.629858    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w" event={"ID":"6e70ec99-2141-4616-b598-84675558b246","Type":"ContainerStarted","Data":"590be24803e328aa46b17bf2da3e33a645613faee0f88701d0413d07e816fc9a"}
Oct 30 22:15:01.635160 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.635124    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8" event={"ID":"003da65f-5b3a-4735-bf4d-8f064ad48d96","Type":"ContainerStarted","Data":"694e643741747e41aeb125c1daa84b599a3e5b7e1a2153ea2fb386e287b0493a"}
Oct 30 22:15:01.635358 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.635182    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8" event={"ID":"003da65f-5b3a-4735-bf4d-8f064ad48d96","Type":"ContainerStarted","Data":"2ac0d2e9cbbd62a7700180b99155fb9c136b209c067d89818578fdad76be37d4"}
Oct 30 22:15:01.757948 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.757890    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4" podStartSLOduration=1.757876361 podStartE2EDuration="1.757876361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:01.755912821 +0000 UTC m=+2444.427816759" watchObservedRunningTime="2026-10-30 22:15:01.757876361 +0000 UTC m=+2444.429780295"
Oct 30 22:15:01.889116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.889006    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w" podStartSLOduration=1.888989596 podStartE2EDuration="1.888989596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:01.885228212 +0000 UTC m=+2444.557132138" watchObservedRunningTime="2026-10-30 22:15:01.888989596 +0000 UTC m=+2444.560893531"
Oct 30 22:15:01.939121 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:01.939085    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:02.037765 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.037712    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh" podStartSLOduration=2.037695416 podStartE2EDuration="2.037695416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:02.035647068 +0000 UTC m=+2444.707551004" watchObservedRunningTime="2026-10-30 22:15:02.037695416 +0000 UTC m=+2444.709599375"
Oct 30 22:15:02.180512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.180437    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8" podStartSLOduration=2.18041659 podStartE2EDuration="2.18041659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:02.176206776 +0000 UTC m=+2444.848110712" watchObservedRunningTime="2026-10-30 22:15:02.18041659 +0000 UTC m=+2444.852320525"
Oct 30 22:15:02.284968 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.284924    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:02.505347 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.505151    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:02.652540 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.652493    5586 generic.go:356] "Generic (PLEG): container finished" podID="7ad8b80f-746c-4450-a732-3551846186c7" containerID="1299760c09be9ac803dab8ef53096316c15dd4304621dc29ac67e1814c45b7c0" exitCode=60
Oct 30 22:15:02.653301 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.652557    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4" event={"ID":"7ad8b80f-746c-4450-a732-3551846186c7","Type":"ContainerDied","Data":"1299760c09be9ac803dab8ef53096316c15dd4304621dc29ac67e1814c45b7c0"}
Oct 30 22:15:02.653301 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.653012    5586 scope.go:122] "RemoveContainer" containerID="1299760c09be9ac803dab8ef53096316c15dd4304621dc29ac67e1814c45b7c0"
Oct 30 22:15:02.655144 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.655038    5586 generic.go:356] "Generic (PLEG): container finished" podID="07444e4a-f484-4ff4-bffd-9c3f329fca00" containerID="79120f91a1b2ab74ba247b9403c37415c168bcd22a0f45bc26c60d01142f6b6f" exitCode=60
Oct 30 22:15:02.655144 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.655078    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh" event={"ID":"07444e4a-f484-4ff4-bffd-9c3f329fca00","Type":"ContainerDied","Data":"79120f91a1b2ab74ba247b9403c37415c168bcd22a0f45bc26c60d01142f6b6f"}
Oct 30 22:15:02.655421 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.655399    5586 scope.go:122] "RemoveContainer" containerID="79120f91a1b2ab74ba247b9403c37415c168bcd22a0f45bc26c60d01142f6b6f"
Oct 30 22:15:02.658345 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.658317    5586 generic.go:356] "Generic (PLEG): container finished" podID="6e70ec99-2141-4616-b598-84675558b246" containerID="45ecf3b7243e69002100205ff2c64a32d32ae39cdbf9eccd6241ec605e35b547" exitCode=60
Oct 30 22:15:02.658643 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.658361    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w" event={"ID":"6e70ec99-2141-4616-b598-84675558b246","Type":"ContainerDied","Data":"45ecf3b7243e69002100205ff2c64a32d32ae39cdbf9eccd6241ec605e35b547"}
Oct 30 22:15:02.658729 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.658686    5586 scope.go:122] "RemoveContainer" containerID="45ecf3b7243e69002100205ff2c64a32d32ae39cdbf9eccd6241ec605e35b547"
Oct 30 22:15:02.660522 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.660497    5586 generic.go:356] "Generic (PLEG): container finished" podID="003da65f-5b3a-4735-bf4d-8f064ad48d96" containerID="694e643741747e41aeb125c1daa84b599a3e5b7e1a2153ea2fb386e287b0493a" exitCode=60
Oct 30 22:15:02.660630 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.660580    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8" event={"ID":"003da65f-5b3a-4735-bf4d-8f064ad48d96","Type":"ContainerDied","Data":"694e643741747e41aeb125c1daa84b599a3e5b7e1a2153ea2fb386e287b0493a"}
Oct 30 22:15:02.660771 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.660757    5586 scope.go:122] "RemoveContainer" containerID="694e643741747e41aeb125c1daa84b599a3e5b7e1a2153ea2fb386e287b0493a"
Oct 30 22:15:02.680913 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:02.679396    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:03.332466 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:03.332428    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:03.714257 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:03.713996    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w" event={"ID":"6e70ec99-2141-4616-b598-84675558b246","Type":"ContainerStarted","Data":"7bf9ac4d3d0420484432cef4513656d265567d9bec99cfc12f46382620523266"}
Oct 30 22:15:03.719131 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:03.718440    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8" event={"ID":"003da65f-5b3a-4735-bf4d-8f064ad48d96","Type":"ContainerStarted","Data":"b09169b8fefcd0bf1098dce624d046c2300f9d6fe99cbb0c88916e54f880c3eb"}
Oct 30 22:15:03.726115 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:03.725560    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4" event={"ID":"7ad8b80f-746c-4450-a732-3551846186c7","Type":"ContainerStarted","Data":"cdb7761840691f0f5132c6197cb4364e5bc062d90abcf49670787af856d58d5a"}
Oct 30 22:15:03.735017 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:03.734975    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh" event={"ID":"07444e4a-f484-4ff4-bffd-9c3f329fca00","Type":"ContainerStarted","Data":"b764c4f73458c6c2d77d55432712c2dc25c49e8c62e56986d9bcffe7def2ef44"}
Oct 30 22:15:04.743547 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.743505    5586 generic.go:356] "Generic (PLEG): container finished" podID="7ad8b80f-746c-4450-a732-3551846186c7" containerID="cdb7761840691f0f5132c6197cb4364e5bc062d90abcf49670787af856d58d5a" exitCode=60
Oct 30 22:15:04.744045 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.743591    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4" event={"ID":"7ad8b80f-746c-4450-a732-3551846186c7","Type":"ContainerDied","Data":"cdb7761840691f0f5132c6197cb4364e5bc062d90abcf49670787af856d58d5a"}
Oct 30 22:15:04.744045 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.743662    5586 scope.go:122] "RemoveContainer" containerID="1299760c09be9ac803dab8ef53096316c15dd4304621dc29ac67e1814c45b7c0"
Oct 30 22:15:04.744045 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.743911    5586 scope.go:122] "RemoveContainer" containerID="cdb7761840691f0f5132c6197cb4364e5bc062d90abcf49670787af856d58d5a"
Oct 30 22:15:04.744303 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:04.744283    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4_odh-ai-gateway-infra(7ad8b80f-746c-4450-a732-3551846186c7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4" podUID="7ad8b80f-746c-4450-a732-3551846186c7"
Oct 30 22:15:04.745865 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.745841    5586 generic.go:356] "Generic (PLEG): container finished" podID="07444e4a-f484-4ff4-bffd-9c3f329fca00" containerID="b764c4f73458c6c2d77d55432712c2dc25c49e8c62e56986d9bcffe7def2ef44" exitCode=60
Oct 30 22:15:04.745951 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.745890    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh" event={"ID":"07444e4a-f484-4ff4-bffd-9c3f329fca00","Type":"ContainerDied","Data":"b764c4f73458c6c2d77d55432712c2dc25c49e8c62e56986d9bcffe7def2ef44"}
Oct 30 22:15:04.746222 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.746205    5586 scope.go:122] "RemoveContainer" containerID="b764c4f73458c6c2d77d55432712c2dc25c49e8c62e56986d9bcffe7def2ef44"
Oct 30 22:15:04.746533 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:04.746513    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh_odh-ai-gateway-infra(07444e4a-f484-4ff4-bffd-9c3f329fca00)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh" podUID="07444e4a-f484-4ff4-bffd-9c3f329fca00"
Oct 30 22:15:04.747828 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.747807    5586 generic.go:356] "Generic (PLEG): container finished" podID="6e70ec99-2141-4616-b598-84675558b246" containerID="7bf9ac4d3d0420484432cef4513656d265567d9bec99cfc12f46382620523266" exitCode=60
Oct 30 22:15:04.747916 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.747886    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w" event={"ID":"6e70ec99-2141-4616-b598-84675558b246","Type":"ContainerDied","Data":"7bf9ac4d3d0420484432cef4513656d265567d9bec99cfc12f46382620523266"}
Oct 30 22:15:04.748123 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.748106    5586 scope.go:122] "RemoveContainer" containerID="7bf9ac4d3d0420484432cef4513656d265567d9bec99cfc12f46382620523266"
Oct 30 22:15:04.748456 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:04.748434    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w_odh-ai-gateway-infra(6e70ec99-2141-4616-b598-84675558b246)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w" podUID="6e70ec99-2141-4616-b598-84675558b246"
Oct 30 22:15:04.749781 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.749761    5586 generic.go:356] "Generic (PLEG): container finished" podID="003da65f-5b3a-4735-bf4d-8f064ad48d96" containerID="b09169b8fefcd0bf1098dce624d046c2300f9d6fe99cbb0c88916e54f880c3eb" exitCode=60
Oct 30 22:15:04.749861 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.749832    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8" event={"ID":"003da65f-5b3a-4735-bf4d-8f064ad48d96","Type":"ContainerDied","Data":"b09169b8fefcd0bf1098dce624d046c2300f9d6fe99cbb0c88916e54f880c3eb"}
Oct 30 22:15:04.749989 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.749976    5586 scope.go:122] "RemoveContainer" containerID="b09169b8fefcd0bf1098dce624d046c2300f9d6fe99cbb0c88916e54f880c3eb"
Oct 30 22:15:04.750170 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:04.750154    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8_odh-ai-gateway-infra(003da65f-5b3a-4735-bf4d-8f064ad48d96)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8" podUID="003da65f-5b3a-4735-bf4d-8f064ad48d96"
Oct 30 22:15:04.756843 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.756812    5586 scope.go:122] "RemoveContainer" containerID="79120f91a1b2ab74ba247b9403c37415c168bcd22a0f45bc26c60d01142f6b6f"
Oct 30 22:15:04.770967 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.770947    5586 scope.go:122] "RemoveContainer" containerID="45ecf3b7243e69002100205ff2c64a32d32ae39cdbf9eccd6241ec605e35b547"
Oct 30 22:15:04.784282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:04.784261    5586 scope.go:122] "RemoveContainer" containerID="694e643741747e41aeb125c1daa84b599a3e5b7e1a2153ea2fb386e287b0493a"
Oct 30 22:15:05.760170 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:05.760141    5586 scope.go:122] "RemoveContainer" containerID="cdb7761840691f0f5132c6197cb4364e5bc062d90abcf49670787af856d58d5a"
Oct 30 22:15:05.760686 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:05.760540    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4_odh-ai-gateway-infra(7ad8b80f-746c-4450-a732-3551846186c7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4" podUID="7ad8b80f-746c-4450-a732-3551846186c7"
Oct 30 22:15:05.761698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:05.761672    5586 scope.go:122] "RemoveContainer" containerID="b764c4f73458c6c2d77d55432712c2dc25c49e8c62e56986d9bcffe7def2ef44"
Oct 30 22:15:05.761993 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:05.761968    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh_odh-ai-gateway-infra(07444e4a-f484-4ff4-bffd-9c3f329fca00)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh" podUID="07444e4a-f484-4ff4-bffd-9c3f329fca00"
Oct 30 22:15:05.763351 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:05.763328    5586 scope.go:122] "RemoveContainer" containerID="7bf9ac4d3d0420484432cef4513656d265567d9bec99cfc12f46382620523266"
Oct 30 22:15:05.763618 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:05.763594    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w_odh-ai-gateway-infra(6e70ec99-2141-4616-b598-84675558b246)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w" podUID="6e70ec99-2141-4616-b598-84675558b246"
Oct 30 22:15:05.764980 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:05.764964    5586 scope.go:122] "RemoveContainer" containerID="b09169b8fefcd0bf1098dce624d046c2300f9d6fe99cbb0c88916e54f880c3eb"
Oct 30 22:15:05.765202 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:05.765184    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8_odh-ai-gateway-infra(003da65f-5b3a-4735-bf4d-8f064ad48d96)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8" podUID="003da65f-5b3a-4735-bf4d-8f064ad48d96"
Oct 30 22:15:07.601484 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:07.601432    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:08.751387 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.751358    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"
Oct 30 22:15:08.801770 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.801735    5586 generic.go:356] "Generic (PLEG): container finished" podID="2d4ab435-292c-4532-a4de-412d74acc5ae" containerID="9c6fdba8a0f37b44706ee7aad0f8d908c8cfc31c2543411f0071c952c3918a9f" exitCode=0
Oct 30 22:15:08.801968 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.801811    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx" event={"ID":"2d4ab435-292c-4532-a4de-412d74acc5ae","Type":"ContainerDied","Data":"9c6fdba8a0f37b44706ee7aad0f8d908c8cfc31c2543411f0071c952c3918a9f"}
Oct 30 22:15:08.803808 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.803781    5586 generic.go:356] "Generic (PLEG): container finished" podID="a0622aae-1702-4cb1-80b6-8740c5261667" containerID="18b92ea91d1fdaf4539ad1e91021dde2b92320eb37de3472069efa1eded9f3dc" exitCode=0
Oct 30 22:15:08.803951 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.803817    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh" event={"ID":"a0622aae-1702-4cb1-80b6-8740c5261667","Type":"ContainerDied","Data":"18b92ea91d1fdaf4539ad1e91021dde2b92320eb37de3472069efa1eded9f3dc"}
Oct 30 22:15:08.803951 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.803847    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"
Oct 30 22:15:08.803951 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.803862    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh" event={"ID":"a0622aae-1702-4cb1-80b6-8740c5261667","Type":"ContainerDied","Data":"46c9c22e53aee14281cf3b03c7e45f949a6a613008374262843215a806032205"}
Oct 30 22:15:08.803951 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.803886    5586 scope.go:122] "RemoveContainer" containerID="18b92ea91d1fdaf4539ad1e91021dde2b92320eb37de3472069efa1eded9f3dc"
Oct 30 22:15:08.817675 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.817652    5586 scope.go:122] "RemoveContainer" containerID="18b92ea91d1fdaf4539ad1e91021dde2b92320eb37de3472069efa1eded9f3dc"
Oct 30 22:15:08.818109 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:08.818078    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18b92ea91d1fdaf4539ad1e91021dde2b92320eb37de3472069efa1eded9f3dc\": container with ID starting with 18b92ea91d1fdaf4539ad1e91021dde2b92320eb37de3472069efa1eded9f3dc not found: ID does not exist" containerID="18b92ea91d1fdaf4539ad1e91021dde2b92320eb37de3472069efa1eded9f3dc"
Oct 30 22:15:08.818225 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.818120    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18b92ea91d1fdaf4539ad1e91021dde2b92320eb37de3472069efa1eded9f3dc"} err="failed to get container status \"18b92ea91d1fdaf4539ad1e91021dde2b92320eb37de3472069efa1eded9f3dc\": rpc error: code = NotFound desc = could not find container \"18b92ea91d1fdaf4539ad1e91021dde2b92320eb37de3472069efa1eded9f3dc\": container with ID starting with 18b92ea91d1fdaf4539ad1e91021dde2b92320eb37de3472069efa1eded9f3dc not found: ID does not exist"
Oct 30 22:15:08.839965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.839817    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0622aae-1702-4cb1-80b6-8740c5261667-kube-api-access-7zggn\" (UniqueName: \"kubernetes.io/projected/a0622aae-1702-4cb1-80b6-8740c5261667-kube-api-access-7zggn\") pod \"a0622aae-1702-4cb1-80b6-8740c5261667\" (UID: \"a0622aae-1702-4cb1-80b6-8740c5261667\") "
Oct 30 22:15:08.839965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.839955    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a0622aae-1702-4cb1-80b6-8740c5261667-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0622aae-1702-4cb1-80b6-8740c5261667-plugins-config-volume\") pod \"a0622aae-1702-4cb1-80b6-8740c5261667\" (UID: \"a0622aae-1702-4cb1-80b6-8740c5261667\") "
Oct 30 22:15:08.840593 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.840564    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0622aae-1702-4cb1-80b6-8740c5261667-plugins-config-volume" pod "a0622aae-1702-4cb1-80b6-8740c5261667" (UID: "a0622aae-1702-4cb1-80b6-8740c5261667"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:08.842272 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.842224    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0622aae-1702-4cb1-80b6-8740c5261667-kube-api-access-7zggn" pod "a0622aae-1702-4cb1-80b6-8740c5261667" (UID: "a0622aae-1702-4cb1-80b6-8740c5261667"). InnerVolumeSpecName "kube-api-access-7zggn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:08.853503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.853478    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"
Oct 30 22:15:08.941639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.941585    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d4ab435-292c-4532-a4de-412d74acc5ae-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4ab435-292c-4532-a4de-412d74acc5ae-plugins-config-volume\") pod \"2d4ab435-292c-4532-a4de-412d74acc5ae\" (UID: \"2d4ab435-292c-4532-a4de-412d74acc5ae\") "
Oct 30 22:15:08.941853 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.941834    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d4ab435-292c-4532-a4de-412d74acc5ae-kube-api-access-kqxgr\" (UniqueName: \"kubernetes.io/projected/2d4ab435-292c-4532-a4de-412d74acc5ae-kube-api-access-kqxgr\") pod \"2d4ab435-292c-4532-a4de-412d74acc5ae\" (UID: \"2d4ab435-292c-4532-a4de-412d74acc5ae\") "
Oct 30 22:15:08.941995 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.941969    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d4ab435-292c-4532-a4de-412d74acc5ae-plugins-config-volume" pod "2d4ab435-292c-4532-a4de-412d74acc5ae" (UID: "2d4ab435-292c-4532-a4de-412d74acc5ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:08.942353 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.942331    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zggn\" (UniqueName: \"kubernetes.io/projected/a0622aae-1702-4cb1-80b6-8740c5261667-kube-api-access-7zggn\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:08.942452 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.942356    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4ab435-292c-4532-a4de-412d74acc5ae-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:08.942452 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.942370    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0622aae-1702-4cb1-80b6-8740c5261667-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:08.943948 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:08.943922    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d4ab435-292c-4532-a4de-412d74acc5ae-kube-api-access-kqxgr" pod "2d4ab435-292c-4532-a4de-412d74acc5ae" (UID: "2d4ab435-292c-4532-a4de-412d74acc5ae"). InnerVolumeSpecName "kube-api-access-kqxgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:09.043508 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:09.043405    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqxgr\" (UniqueName: \"kubernetes.io/projected/2d4ab435-292c-4532-a4de-412d74acc5ae-kube-api-access-kqxgr\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:09.225095 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:09.224783    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"]
Oct 30 22:15:09.228151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:09.228126    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c79cb5-57d6bdb6d5-6x7jh"]
Oct 30 22:15:09.449278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:09.449225    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0622aae-1702-4cb1-80b6-8740c5261667" path="/var/lib/kubelet/pods/a0622aae-1702-4cb1-80b6-8740c5261667/volumes"
Oct 30 22:15:09.813385 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:09.813300    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"
Oct 30 22:15:09.813385 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:09.813339    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx" event={"ID":"2d4ab435-292c-4532-a4de-412d74acc5ae","Type":"ContainerDied","Data":"386680365c9741a344b3756016997035aea45519130663fc71f1d95950e6e341"}
Oct 30 22:15:09.813915 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:09.813410    5586 scope.go:122] "RemoveContainer" containerID="9c6fdba8a0f37b44706ee7aad0f8d908c8cfc31c2543411f0071c952c3918a9f"
Oct 30 22:15:09.935537 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:09.935491    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:09.979717 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:09.979675    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"]
Oct 30 22:15:10.028850 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:10.028814    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c79cb5-b9bdf965f-kstkx"]
Oct 30 22:15:10.104950 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:10.104845    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:10.209526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:10.209490    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"]
Oct 30 22:15:10.209738 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:10.209715    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf" podUID="37583645-5073-494b-8079-7eac9ceb5d30" containerName="istio-proxy" containerID="cri-o://595518896f12d3881879831fe0a1ad923f36e5ff30b6e49ea1f59034cb47e9cd" gracePeriod=30
Oct 30 22:15:10.280042 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:10.280006    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:10.816506 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:10.816468    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:10.985350 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:10.985311    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:11.168918 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:11.168877    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:11.449421 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:11.449309    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d4ab435-292c-4532-a4de-412d74acc5ae" path="/var/lib/kubelet/pods/2d4ab435-292c-4532-a4de-412d74acc5ae/volumes"
Oct 30 22:15:12.807619 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:12.807581    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:13.995274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:13.993535    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:15.003133 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.003093    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:15.213970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.206343    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:15.440637 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.440596    5586 scope.go:122] "RemoveContainer" containerID="7bf9ac4d3d0420484432cef4513656d265567d9bec99cfc12f46382620523266"
Oct 30 22:15:15.440821 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.440799    5586 scope.go:122] "RemoveContainer" containerID="b764c4f73458c6c2d77d55432712c2dc25c49e8c62e56986d9bcffe7def2ef44"
Oct 30 22:15:15.490153 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.490125    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 22:15:15.543794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.543759    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-workload-certs\") pod \"37583645-5073-494b-8079-7eac9ceb5d30\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") "
Oct 30 22:15:15.543794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.543801    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37583645-5073-494b-8079-7eac9ceb5d30-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37583645-5073-494b-8079-7eac9ceb5d30-istiod-ca-cert\") pod \"37583645-5073-494b-8079-7eac9ceb5d30\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") "
Oct 30 22:15:15.544005 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.543955    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37583645-5073-494b-8079-7eac9ceb5d30-kube-api-access-6lqkf\" (UniqueName: \"kubernetes.io/projected/37583645-5073-494b-8079-7eac9ceb5d30-kube-api-access-6lqkf\") pod \"37583645-5073-494b-8079-7eac9ceb5d30\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") "
Oct 30 22:15:15.544049 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.544014    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/37583645-5073-494b-8079-7eac9ceb5d30-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37583645-5073-494b-8079-7eac9ceb5d30-istio-podinfo\") pod \"37583645-5073-494b-8079-7eac9ceb5d30\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") "
Oct 30 22:15:15.544082 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.544044    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-credential-socket\") pod \"37583645-5073-494b-8079-7eac9ceb5d30\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") "
Oct 30 22:15:15.544130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.544078    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-workload-certs" pod "37583645-5073-494b-8079-7eac9ceb5d30" (UID: "37583645-5073-494b-8079-7eac9ceb5d30"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:15.544130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.544110    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-istio-data\") pod \"37583645-5073-494b-8079-7eac9ceb5d30\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") "
Oct 30 22:15:15.544274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.544139    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37583645-5073-494b-8079-7eac9ceb5d30-istio-token\" (UniqueName: \"kubernetes.io/projected/37583645-5073-494b-8079-7eac9ceb5d30-istio-token\") pod \"37583645-5073-494b-8079-7eac9ceb5d30\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") "
Oct 30 22:15:15.544274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.544167    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-workload-socket\") pod \"37583645-5073-494b-8079-7eac9ceb5d30\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") "
Oct 30 22:15:15.544398 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.544311    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37583645-5073-494b-8079-7eac9ceb5d30-istiod-ca-cert" pod "37583645-5073-494b-8079-7eac9ceb5d30" (UID: "37583645-5073-494b-8079-7eac9ceb5d30"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:15.544480 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.544451    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-credential-socket" pod "37583645-5073-494b-8079-7eac9ceb5d30" (UID: "37583645-5073-494b-8079-7eac9ceb5d30"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:15.544544 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.544473    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-istio-data" pod "37583645-5073-494b-8079-7eac9ceb5d30" (UID: "37583645-5073-494b-8079-7eac9ceb5d30"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:15.544544 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.544489    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-workload-socket" pod "37583645-5073-494b-8079-7eac9ceb5d30" (UID: "37583645-5073-494b-8079-7eac9ceb5d30"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:15.544544 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.544501    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-istio-envoy\") pod \"37583645-5073-494b-8079-7eac9ceb5d30\" (UID: \"37583645-5073-494b-8079-7eac9ceb5d30\") "
Oct 30 22:15:15.545365 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.545336    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.545506 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.545369    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.545506 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.545384    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.545506 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.545398    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.545506 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.545413    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37583645-5073-494b-8079-7eac9ceb5d30-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.623662 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.623574    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37583645-5073-494b-8079-7eac9ceb5d30-istio-podinfo" pod "37583645-5073-494b-8079-7eac9ceb5d30" (UID: "37583645-5073-494b-8079-7eac9ceb5d30"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:15:15.623834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.623673    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37583645-5073-494b-8079-7eac9ceb5d30-kube-api-access-6lqkf" pod "37583645-5073-494b-8079-7eac9ceb5d30" (UID: "37583645-5073-494b-8079-7eac9ceb5d30"). InnerVolumeSpecName "kube-api-access-6lqkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:15.624031 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.624007    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-istio-envoy" pod "37583645-5073-494b-8079-7eac9ceb5d30" (UID: "37583645-5073-494b-8079-7eac9ceb5d30"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:15.624464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.624437    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37583645-5073-494b-8079-7eac9ceb5d30-istio-token" pod "37583645-5073-494b-8079-7eac9ceb5d30" (UID: "37583645-5073-494b-8079-7eac9ceb5d30"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:15.646901 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.646867    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lqkf\" (UniqueName: \"kubernetes.io/projected/37583645-5073-494b-8079-7eac9ceb5d30-kube-api-access-6lqkf\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.647051 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.646904    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37583645-5073-494b-8079-7eac9ceb5d30-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.647051 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.646924    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37583645-5073-494b-8079-7eac9ceb5d30-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.647051 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.646940    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37583645-5073-494b-8079-7eac9ceb5d30-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.868402 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.868365    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w" event={"ID":"6e70ec99-2141-4616-b598-84675558b246","Type":"ContainerStarted","Data":"731e11013c404045240ba53070f55e2223ffb3896ab1b649109515ba1226b3f8"}
Oct 30 22:15:15.870114 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.870088    5586 generic.go:356] "Generic (PLEG): container finished" podID="37583645-5073-494b-8079-7eac9ceb5d30" containerID="595518896f12d3881879831fe0a1ad923f36e5ff30b6e49ea1f59034cb47e9cd" exitCode=0
Oct 30 22:15:15.870276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.870127    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"
Oct 30 22:15:15.870276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.870165    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf" event={"ID":"37583645-5073-494b-8079-7eac9ceb5d30","Type":"ContainerDied","Data":"595518896f12d3881879831fe0a1ad923f36e5ff30b6e49ea1f59034cb47e9cd"}
Oct 30 22:15:15.870276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.870218    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf" event={"ID":"37583645-5073-494b-8079-7eac9ceb5d30","Type":"ContainerDied","Data":"4de8192824b9f9d993604332f60b31963ce7b29ae953d6eda407b6a963d9646e"}
Oct 30 22:15:15.870276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.870264    5586 scope.go:122] "RemoveContainer" containerID="595518896f12d3881879831fe0a1ad923f36e5ff30b6e49ea1f59034cb47e9cd"
Oct 30 22:15:15.872602 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.872578    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh" event={"ID":"07444e4a-f484-4ff4-bffd-9c3f329fca00","Type":"ContainerStarted","Data":"00a2f3dc62763e03e7c72241270cb0a2034fdf0612f3fc807b816ecac6c3b338"}
Oct 30 22:15:15.881280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.881225    5586 scope.go:122] "RemoveContainer" containerID="1e5933086ae37e06e5887a52e8c3336a2e5e5992bbca81f0c5c22f870da44bb0"
Oct 30 22:15:15.891599 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.891576    5586 scope.go:122] "RemoveContainer" containerID="595518896f12d3881879831fe0a1ad923f36e5ff30b6e49ea1f59034cb47e9cd"
Oct 30 22:15:15.891930 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:15.891908    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"595518896f12d3881879831fe0a1ad923f36e5ff30b6e49ea1f59034cb47e9cd\": container with ID starting with 595518896f12d3881879831fe0a1ad923f36e5ff30b6e49ea1f59034cb47e9cd not found: ID does not exist" containerID="595518896f12d3881879831fe0a1ad923f36e5ff30b6e49ea1f59034cb47e9cd"
Oct 30 22:15:15.892018 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.891938    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595518896f12d3881879831fe0a1ad923f36e5ff30b6e49ea1f59034cb47e9cd"} err="failed to get container status \"595518896f12d3881879831fe0a1ad923f36e5ff30b6e49ea1f59034cb47e9cd\": rpc error: code = NotFound desc = could not find container \"595518896f12d3881879831fe0a1ad923f36e5ff30b6e49ea1f59034cb47e9cd\": container with ID starting with 595518896f12d3881879831fe0a1ad923f36e5ff30b6e49ea1f59034cb47e9cd not found: ID does not exist"
Oct 30 22:15:15.892018 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.891957    5586 scope.go:122] "RemoveContainer" containerID="1e5933086ae37e06e5887a52e8c3336a2e5e5992bbca81f0c5c22f870da44bb0"
Oct 30 22:15:15.892185 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:15.892170    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e5933086ae37e06e5887a52e8c3336a2e5e5992bbca81f0c5c22f870da44bb0\": container with ID starting with 1e5933086ae37e06e5887a52e8c3336a2e5e5992bbca81f0c5c22f870da44bb0 not found: ID does not exist" containerID="1e5933086ae37e06e5887a52e8c3336a2e5e5992bbca81f0c5c22f870da44bb0"
Oct 30 22:15:15.892244 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:15.892190    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e5933086ae37e06e5887a52e8c3336a2e5e5992bbca81f0c5c22f870da44bb0"} err="failed to get container status \"1e5933086ae37e06e5887a52e8c3336a2e5e5992bbca81f0c5c22f870da44bb0\": rpc error: code = NotFound desc = could not find container \"1e5933086ae37e06e5887a52e8c3336a2e5e5992bbca81f0c5c22f870da44bb0\": container with ID starting with 1e5933086ae37e06e5887a52e8c3336a2e5e5992bbca81f0c5c22f870da44bb0 not found: ID does not exist"
Oct 30 22:15:16.113112 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:16.113068    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"]
Oct 30 22:15:16.117471 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:16.117441    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-557708-openshift-default-bd7dbddd5-jbqqf"]
Oct 30 22:15:16.441105 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:16.441076    5586 scope.go:122] "RemoveContainer" containerID="b09169b8fefcd0bf1098dce624d046c2300f9d6fe99cbb0c88916e54f880c3eb"
Oct 30 22:15:16.881665 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:16.881633    5586 generic.go:356] "Generic (PLEG): container finished" podID="07444e4a-f484-4ff4-bffd-9c3f329fca00" containerID="00a2f3dc62763e03e7c72241270cb0a2034fdf0612f3fc807b816ecac6c3b338" exitCode=60
Oct 30 22:15:16.881832 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:16.881702    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh" event={"ID":"07444e4a-f484-4ff4-bffd-9c3f329fca00","Type":"ContainerDied","Data":"00a2f3dc62763e03e7c72241270cb0a2034fdf0612f3fc807b816ecac6c3b338"}
Oct 30 22:15:16.881832 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:16.881767    5586 scope.go:122] "RemoveContainer" containerID="b764c4f73458c6c2d77d55432712c2dc25c49e8c62e56986d9bcffe7def2ef44"
Oct 30 22:15:16.882256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:16.882221    5586 scope.go:122] "RemoveContainer" containerID="00a2f3dc62763e03e7c72241270cb0a2034fdf0612f3fc807b816ecac6c3b338"
Oct 30 22:15:16.882637 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:16.882617    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh_odh-ai-gateway-infra(07444e4a-f484-4ff4-bffd-9c3f329fca00)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh" podUID="07444e4a-f484-4ff4-bffd-9c3f329fca00"
Oct 30 22:15:16.884011 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:16.883983    5586 generic.go:356] "Generic (PLEG): container finished" podID="6e70ec99-2141-4616-b598-84675558b246" containerID="731e11013c404045240ba53070f55e2223ffb3896ab1b649109515ba1226b3f8" exitCode=60
Oct 30 22:15:16.884122 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:16.884031    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w" event={"ID":"6e70ec99-2141-4616-b598-84675558b246","Type":"ContainerDied","Data":"731e11013c404045240ba53070f55e2223ffb3896ab1b649109515ba1226b3f8"}
Oct 30 22:15:16.884481 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:16.884463    5586 scope.go:122] "RemoveContainer" containerID="731e11013c404045240ba53070f55e2223ffb3896ab1b649109515ba1226b3f8"
Oct 30 22:15:16.884806 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:16.884781    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w_odh-ai-gateway-infra(6e70ec99-2141-4616-b598-84675558b246)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w" podUID="6e70ec99-2141-4616-b598-84675558b246"
Oct 30 22:15:16.887288 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:16.887263    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8" event={"ID":"003da65f-5b3a-4735-bf4d-8f064ad48d96","Type":"ContainerStarted","Data":"86daab8035b425e2a8e46489ab2261c6048f179f7b4eb13e65894ea990cb95f9"}
Oct 30 22:15:16.895211 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:16.895190    5586 scope.go:122] "RemoveContainer" containerID="7bf9ac4d3d0420484432cef4513656d265567d9bec99cfc12f46382620523266"
Oct 30 22:15:17.007573 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.007490    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"]
Oct 30 22:15:17.260338 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.260246    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"]
Oct 30 22:15:17.290147 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.290109    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"]
Oct 30 22:15:17.291074 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.291050    5586 cpu_manager.go:414] "containerMap: removing container" podUID="2d4ab435-292c-4532-a4de-412d74acc5ae" containerName="payload-pre-processing"
Oct 30 22:15:17.291123 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.291079    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d4ab435-292c-4532-a4de-412d74acc5ae" containerName="payload-pre-processing"
Oct 30 22:15:17.291123 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.291095    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a0622aae-1702-4cb1-80b6-8740c5261667" containerName="payload-processing"
Oct 30 22:15:17.291123 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.291103    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0622aae-1702-4cb1-80b6-8740c5261667" containerName="payload-processing"
Oct 30 22:15:17.291264 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.291121    5586 cpu_manager.go:414] "containerMap: removing container" podUID="37583645-5073-494b-8079-7eac9ceb5d30" containerName="istio-proxy"
Oct 30 22:15:17.291264 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.291131    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37583645-5073-494b-8079-7eac9ceb5d30" containerName="istio-proxy"
Oct 30 22:15:17.291362 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.291346    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d4ab435-292c-4532-a4de-412d74acc5ae" containerName="payload-pre-processing"
Oct 30 22:15:17.291405 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.291366    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0622aae-1702-4cb1-80b6-8740c5261667" containerName="payload-processing"
Oct 30 22:15:17.291405 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.291378    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="37583645-5073-494b-8079-7eac9ceb5d30" containerName="istio-proxy"
Oct 30 22:15:17.291405 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.291391    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="37583645-5073-494b-8079-7eac9ceb5d30" containerName="istio-proxy"
Oct 30 22:15:17.296639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.296615    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"
Oct 30 22:15:17.361613 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.361581    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"]
Oct 30 22:15:17.375597 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.375560    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-735132ea-v9llp\" (UID: \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"
Oct 30 22:15:17.375809 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.375776    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-735132ea-v9llp\" (UID: \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"
Oct 30 22:15:17.375884 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.375823    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tq6b\" (UniqueName: \"kubernetes.io/projected/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-kube-api-access-6tq6b\") pod \"maas-api-revoke-sub-735132ea-v9llp\" (UID: \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"
Oct 30 22:15:17.383012 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.382982    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:15:17.440541 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.440513    5586 scope.go:122] "RemoveContainer" containerID="cdb7761840691f0f5132c6197cb4364e5bc062d90abcf49670787af856d58d5a"
Oct 30 22:15:17.449452 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.449421    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37583645-5073-494b-8079-7eac9ceb5d30" path="/var/lib/kubelet/pods/37583645-5073-494b-8079-7eac9ceb5d30/volumes"
Oct 30 22:15:17.477692 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.477651    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-735132ea-v9llp\" (UID: \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"
Oct 30 22:15:17.477904 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.477738    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tq6b\" (UniqueName: \"kubernetes.io/projected/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-kube-api-access-6tq6b\") pod \"maas-api-revoke-sub-735132ea-v9llp\" (UID: \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"
Oct 30 22:15:17.477904 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.477805    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-735132ea-v9llp\" (UID: \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"
Oct 30 22:15:17.478487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.478462    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-735132ea-v9llp\" (UID: \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"
Oct 30 22:15:17.529133 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.529054    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:15:17.541334 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.541301    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-735132ea-v9llp\" (UID: \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"
Oct 30 22:15:17.546819 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.546770    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tq6b\" (UniqueName: \"kubernetes.io/projected/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-kube-api-access-6tq6b\") pod \"maas-api-revoke-sub-735132ea-v9llp\" (UID: \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"
Oct 30 22:15:17.612802 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.612771    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"
Oct 30 22:15:17.613218 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.613184    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:17.896324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.896205    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4" event={"ID":"7ad8b80f-746c-4450-a732-3551846186c7","Type":"ContainerStarted","Data":"c1a0e620846017ab485b2cf528a321d284f8b508247f959261ec34d759cb27df"}
Oct 30 22:15:17.902417 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.902383    5586 generic.go:356] "Generic (PLEG): container finished" podID="003da65f-5b3a-4735-bf4d-8f064ad48d96" containerID="86daab8035b425e2a8e46489ab2261c6048f179f7b4eb13e65894ea990cb95f9" exitCode=60
Oct 30 22:15:17.902629 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.902464    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8" event={"ID":"003da65f-5b3a-4735-bf4d-8f064ad48d96","Type":"ContainerDied","Data":"86daab8035b425e2a8e46489ab2261c6048f179f7b4eb13e65894ea990cb95f9"}
Oct 30 22:15:17.902629 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.902566    5586 scope.go:122] "RemoveContainer" containerID="b09169b8fefcd0bf1098dce624d046c2300f9d6fe99cbb0c88916e54f880c3eb"
Oct 30 22:15:17.903024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:17.903007    5586 scope.go:122] "RemoveContainer" containerID="86daab8035b425e2a8e46489ab2261c6048f179f7b4eb13e65894ea990cb95f9"
Oct 30 22:15:17.903429 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:17.903399    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8_odh-ai-gateway-infra(003da65f-5b3a-4735-bf4d-8f064ad48d96)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8" podUID="003da65f-5b3a-4735-bf4d-8f064ad48d96"
Oct 30 22:15:18.010226 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.010177    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"]
Oct 30 22:15:18.013542 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:15:18.013490    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad3229f_17cf_4b4a_8cdc_459636c6ff7e.slice/crio-56ca02d3d48b4bbbe65a1372c991437269aff7008c945f9a2d6a48217b700fb7 WatchSource:0}: Error finding container 56ca02d3d48b4bbbe65a1372c991437269aff7008c945f9a2d6a48217b700fb7: Status 404 returned error can't find the container with id 56ca02d3d48b4bbbe65a1372c991437269aff7008c945f9a2d6a48217b700fb7
Oct 30 22:15:18.051987 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.051946    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"
Oct 30 22:15:18.085574 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.085547    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07444e4a-f484-4ff4-bffd-9c3f329fca00-kube-api-access-rdl7z\" (UniqueName: \"kubernetes.io/projected/07444e4a-f484-4ff4-bffd-9c3f329fca00-kube-api-access-rdl7z\") pod \"07444e4a-f484-4ff4-bffd-9c3f329fca00\" (UID: \"07444e4a-f484-4ff4-bffd-9c3f329fca00\") "
Oct 30 22:15:18.085719 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.085645    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07444e4a-f484-4ff4-bffd-9c3f329fca00-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07444e4a-f484-4ff4-bffd-9c3f329fca00-maas-api-service-ca\") pod \"07444e4a-f484-4ff4-bffd-9c3f329fca00\" (UID: \"07444e4a-f484-4ff4-bffd-9c3f329fca00\") "
Oct 30 22:15:18.086014 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.085975    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07444e4a-f484-4ff4-bffd-9c3f329fca00-maas-api-service-ca" pod "07444e4a-f484-4ff4-bffd-9c3f329fca00" (UID: "07444e4a-f484-4ff4-bffd-9c3f329fca00"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:18.086200 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.086182    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07444e4a-f484-4ff4-bffd-9c3f329fca00-maas-api-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:18.087858 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.087836    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07444e4a-f484-4ff4-bffd-9c3f329fca00-kube-api-access-rdl7z" pod "07444e4a-f484-4ff4-bffd-9c3f329fca00" (UID: "07444e4a-f484-4ff4-bffd-9c3f329fca00"). InnerVolumeSpecName "kube-api-access-rdl7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:18.089265 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.089247    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"
Oct 30 22:15:18.179283 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.179188    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:18.187051 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.187011    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e70ec99-2141-4616-b598-84675558b246-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e70ec99-2141-4616-b598-84675558b246-maas-api-service-ca\") pod \"6e70ec99-2141-4616-b598-84675558b246\" (UID: \"6e70ec99-2141-4616-b598-84675558b246\") "
Oct 30 22:15:18.187253 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.187161    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e70ec99-2141-4616-b598-84675558b246-kube-api-access-zr4js\" (UniqueName: \"kubernetes.io/projected/6e70ec99-2141-4616-b598-84675558b246-kube-api-access-zr4js\") pod \"6e70ec99-2141-4616-b598-84675558b246\" (UID: \"6e70ec99-2141-4616-b598-84675558b246\") "
Oct 30 22:15:18.187501 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.187475    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e70ec99-2141-4616-b598-84675558b246-maas-api-service-ca" pod "6e70ec99-2141-4616-b598-84675558b246" (UID: "6e70ec99-2141-4616-b598-84675558b246"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:18.187559 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.187491    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdl7z\" (UniqueName: \"kubernetes.io/projected/07444e4a-f484-4ff4-bffd-9c3f329fca00-kube-api-access-rdl7z\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:18.189361 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.189342    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e70ec99-2141-4616-b598-84675558b246-kube-api-access-zr4js" pod "6e70ec99-2141-4616-b598-84675558b246" (UID: "6e70ec99-2141-4616-b598-84675558b246"). InnerVolumeSpecName "kube-api-access-zr4js". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:18.259992 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.259927    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"]
Oct 30 22:15:18.289024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.288990    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr4js\" (UniqueName: \"kubernetes.io/projected/6e70ec99-2141-4616-b598-84675558b246-kube-api-access-zr4js\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:18.289024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.289023    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e70ec99-2141-4616-b598-84675558b246-maas-api-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:18.362505 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:18.362462    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:15:18.911579 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.911540    5586 generic.go:356] "Generic (PLEG): container finished" podID="7ad8b80f-746c-4450-a732-3551846186c7" containerID="c1a0e620846017ab485b2cf528a321d284f8b508247f959261ec34d759cb27df" exitCode=60
Oct 30 22:15:18.911752 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.911606    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4" event={"ID":"7ad8b80f-746c-4450-a732-3551846186c7","Type":"ContainerDied","Data":"c1a0e620846017ab485b2cf528a321d284f8b508247f959261ec34d759cb27df"}
Oct 30 22:15:18.911752 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.911690    5586 scope.go:122] "RemoveContainer" containerID="cdb7761840691f0f5132c6197cb4364e5bc062d90abcf49670787af856d58d5a"
Oct 30 22:15:18.912124 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.912102    5586 scope.go:122] "RemoveContainer" containerID="c1a0e620846017ab485b2cf528a321d284f8b508247f959261ec34d759cb27df"
Oct 30 22:15:18.912553 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:18.912523    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4_odh-ai-gateway-infra(7ad8b80f-746c-4450-a732-3551846186c7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4" podUID="7ad8b80f-746c-4450-a732-3551846186c7"
Oct 30 22:15:18.913190 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.913170    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh" event={"ID":"07444e4a-f484-4ff4-bffd-9c3f329fca00","Type":"ContainerDied","Data":"e43282c40a9d29471eec3afbadc937fc1653e497a839100e8ee373d363c6f288"}
Oct 30 22:15:18.913310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.913192    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"
Oct 30 22:15:18.914642 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.914618    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"
Oct 30 22:15:18.914747 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.914619    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w" event={"ID":"6e70ec99-2141-4616-b598-84675558b246","Type":"ContainerDied","Data":"590be24803e328aa46b17bf2da3e33a645613faee0f88701d0413d07e816fc9a"}
Oct 30 22:15:18.916548 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.916519    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp" event={"ID":"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e","Type":"ContainerStarted","Data":"93619b6ca7e1fe02d3e9d31ec6a2aa4d43a1271ee73bd04da7bd78aaac8e9609"}
Oct 30 22:15:18.916658 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.916571    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp" event={"ID":"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e","Type":"ContainerStarted","Data":"56ca02d3d48b4bbbe65a1372c991437269aff7008c945f9a2d6a48217b700fb7"}
Oct 30 22:15:18.929367 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.929345    5586 scope.go:122] "RemoveContainer" containerID="00a2f3dc62763e03e7c72241270cb0a2034fdf0612f3fc807b816ecac6c3b338"
Oct 30 22:15:18.987803 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:18.987781    5586 scope.go:122] "RemoveContainer" containerID="731e11013c404045240ba53070f55e2223ffb3896ab1b649109515ba1226b3f8"
Oct 30 22:15:19.068319 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.068288    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"]
Oct 30 22:15:19.076107 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.076081    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"
Oct 30 22:15:19.200815 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.200734    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/003da65f-5b3a-4735-bf4d-8f064ad48d96-kube-api-access-v2xqt\" (UniqueName: \"kubernetes.io/projected/003da65f-5b3a-4735-bf4d-8f064ad48d96-kube-api-access-v2xqt\") pod \"003da65f-5b3a-4735-bf4d-8f064ad48d96\" (UID: \"003da65f-5b3a-4735-bf4d-8f064ad48d96\") "
Oct 30 22:15:19.200954 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.200863    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/003da65f-5b3a-4735-bf4d-8f064ad48d96-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/003da65f-5b3a-4735-bf4d-8f064ad48d96-maas-api-service-ca\") pod \"003da65f-5b3a-4735-bf4d-8f064ad48d96\" (UID: \"003da65f-5b3a-4735-bf4d-8f064ad48d96\") "
Oct 30 22:15:19.201295 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.201266    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/003da65f-5b3a-4735-bf4d-8f064ad48d96-maas-api-service-ca" pod "003da65f-5b3a-4735-bf4d-8f064ad48d96" (UID: "003da65f-5b3a-4735-bf4d-8f064ad48d96"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:19.201398 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.201380    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/003da65f-5b3a-4735-bf4d-8f064ad48d96-maas-api-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:19.202999 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.202976    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/003da65f-5b3a-4735-bf4d-8f064ad48d96-kube-api-access-v2xqt" pod "003da65f-5b3a-4735-bf4d-8f064ad48d96" (UID: "003da65f-5b3a-4735-bf4d-8f064ad48d96"). InnerVolumeSpecName "kube-api-access-v2xqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:19.302099 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.302053    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2xqt\" (UniqueName: \"kubernetes.io/projected/003da65f-5b3a-4735-bf4d-8f064ad48d96-kube-api-access-v2xqt\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:19.390967 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.390903    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp" podStartSLOduration=2.390889487 podStartE2EDuration="2.390889487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:19.264903714 +0000 UTC m=+2461.936807649" watchObservedRunningTime="2026-10-30 22:15:19.390889487 +0000 UTC m=+2462.062793421"
Oct 30 22:15:19.392925 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.392899    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"]
Oct 30 22:15:19.396920 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.396893    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-80c366-29889975-46q5w"]
Oct 30 22:15:19.448327 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.448291    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e70ec99-2141-4616-b598-84675558b246" path="/var/lib/kubelet/pods/6e70ec99-2141-4616-b598-84675558b246/volumes"
Oct 30 22:15:19.515165 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.515114    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"]
Oct 30 22:15:19.518158 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.518099    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-20c7c7-29889975-czbwh"]
Oct 30 22:15:19.934289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.934221    5586 generic.go:356] "Generic (PLEG): container finished" podID="6ad3229f-17cf-4b4a-8cdc-459636c6ff7e" containerID="93619b6ca7e1fe02d3e9d31ec6a2aa4d43a1271ee73bd04da7bd78aaac8e9609" exitCode=0
Oct 30 22:15:19.934289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.934269    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp" event={"ID":"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e","Type":"ContainerDied","Data":"93619b6ca7e1fe02d3e9d31ec6a2aa4d43a1271ee73bd04da7bd78aaac8e9609"}
Oct 30 22:15:19.935818 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.935784    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"
Oct 30 22:15:19.935935 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.935840    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8" event={"ID":"003da65f-5b3a-4735-bf4d-8f064ad48d96","Type":"ContainerDied","Data":"2ac0d2e9cbbd62a7700180b99155fb9c136b209c067d89818578fdad76be37d4"}
Oct 30 22:15:19.935935 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:19.935912    5586 scope.go:122] "RemoveContainer" containerID="86daab8035b425e2a8e46489ab2261c6048f179f7b4eb13e65894ea990cb95f9"
Oct 30 22:15:20.102152 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:20.102124    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"
Oct 30 22:15:20.163934 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:20.163898    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"]
Oct 30 22:15:20.171325 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:20.171286    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c79cb5-29889975-d7gv8"]
Oct 30 22:15:20.215523 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:20.215432    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ad8b80f-746c-4450-a732-3551846186c7-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad8b80f-746c-4450-a732-3551846186c7-maas-api-service-ca\") pod \"7ad8b80f-746c-4450-a732-3551846186c7\" (UID: \"7ad8b80f-746c-4450-a732-3551846186c7\") "
Oct 30 22:15:20.215684 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:20.215617    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ad8b80f-746c-4450-a732-3551846186c7-kube-api-access-drrtx\" (UniqueName: \"kubernetes.io/projected/7ad8b80f-746c-4450-a732-3551846186c7-kube-api-access-drrtx\") pod \"7ad8b80f-746c-4450-a732-3551846186c7\" (UID: \"7ad8b80f-746c-4450-a732-3551846186c7\") "
Oct 30 22:15:20.215822 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:20.215798    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad8b80f-746c-4450-a732-3551846186c7-maas-api-service-ca" pod "7ad8b80f-746c-4450-a732-3551846186c7" (UID: "7ad8b80f-746c-4450-a732-3551846186c7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:20.216099 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:20.216076    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad8b80f-746c-4450-a732-3551846186c7-maas-api-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:20.217833 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:20.217811    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad8b80f-746c-4450-a732-3551846186c7-kube-api-access-drrtx" pod "7ad8b80f-746c-4450-a732-3551846186c7" (UID: "7ad8b80f-746c-4450-a732-3551846186c7"). InnerVolumeSpecName "kube-api-access-drrtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:20.317512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:20.317476    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drrtx\" (UniqueName: \"kubernetes.io/projected/7ad8b80f-746c-4450-a732-3551846186c7-kube-api-access-drrtx\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:20.795487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:20.795450    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:20.944725 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:20.944693    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"
Oct 30 22:15:20.944913 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:20.944723    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4" event={"ID":"7ad8b80f-746c-4450-a732-3551846186c7","Type":"ContainerDied","Data":"edc5c927ae02be70e50ccc07aa47fd95f9b39e459961c8fb277229f952d3e739"}
Oct 30 22:15:20.944913 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:20.944782    5586 scope.go:122] "RemoveContainer" containerID="c1a0e620846017ab485b2cf528a321d284f8b508247f959261ec34d759cb27df"
Oct 30 22:15:20.966213 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:20.966177    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:21.114286 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.114256    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"
Oct 30 22:15:21.116589 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.116565    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"]
Oct 30 22:15:21.165660 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.165623    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-326d76-29889975-lg5z4"]
Oct 30 22:15:21.227918 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.227829    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-maas-api-openshift-ca\") pod \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\" (UID: \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\") "
Oct 30 22:15:21.228069 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.228000    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-kube-api-access-6tq6b\" (UniqueName: \"kubernetes.io/projected/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-kube-api-access-6tq6b\") pod \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\" (UID: \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\") "
Oct 30 22:15:21.228069 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.228038    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-maas-api-xks-ca\") pod \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\" (UID: \"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e\") "
Oct 30 22:15:21.228281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.228254    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-maas-api-openshift-ca" pod "6ad3229f-17cf-4b4a-8cdc-459636c6ff7e" (UID: "6ad3229f-17cf-4b4a-8cdc-459636c6ff7e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:21.228587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.228562    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:21.230367 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.230342    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-maas-api-xks-ca" pod "6ad3229f-17cf-4b4a-8cdc-459636c6ff7e" (UID: "6ad3229f-17cf-4b4a-8cdc-459636c6ff7e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:15:21.230491 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.230398    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-kube-api-access-6tq6b" pod "6ad3229f-17cf-4b4a-8cdc-459636c6ff7e" (UID: "6ad3229f-17cf-4b4a-8cdc-459636c6ff7e"). InnerVolumeSpecName "kube-api-access-6tq6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:21.330197 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.330159    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tq6b\" (UniqueName: \"kubernetes.io/projected/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-kube-api-access-6tq6b\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:21.330197 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.330196    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:21.447904 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.447860    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="003da65f-5b3a-4735-bf4d-8f064ad48d96" path="/var/lib/kubelet/pods/003da65f-5b3a-4735-bf4d-8f064ad48d96/volumes"
Oct 30 22:15:21.448389 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.448369    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07444e4a-f484-4ff4-bffd-9c3f329fca00" path="/var/lib/kubelet/pods/07444e4a-f484-4ff4-bffd-9c3f329fca00/volumes"
Oct 30 22:15:21.448808 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.448789    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ad8b80f-746c-4450-a732-3551846186c7" path="/var/lib/kubelet/pods/7ad8b80f-746c-4450-a732-3551846186c7/volumes"
Oct 30 22:15:21.954802 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.954763    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp" event={"ID":"6ad3229f-17cf-4b4a-8cdc-459636c6ff7e","Type":"ContainerDied","Data":"56ca02d3d48b4bbbe65a1372c991437269aff7008c945f9a2d6a48217b700fb7"}
Oct 30 22:15:21.955000 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.954816    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56ca02d3d48b4bbbe65a1372c991437269aff7008c945f9a2d6a48217b700fb7"
Oct 30 22:15:21.955000 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:21.954815    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"
Oct 30 22:15:22.708289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:22.708251    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:22.875613 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:22.875574    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:23.231397 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:23.231061    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:24.415409 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:24.415372    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:27.980141 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:27.980099    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:29.886609 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:29.886570    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:31.109946 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:31.109905    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"]
Oct 30 22:15:31.110466 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:31.110223    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy" containerID="cri-o://881994a42458a3976ce76c6d80b1856d70db6638c4c2f4457c4fbf2ec417f995" gracePeriod=30
Oct 30 22:15:31.146910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:31.146872    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"]
Oct 30 22:15:31.147142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:31.147117    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc" podUID="dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" containerName="istio-proxy" containerID="cri-o://0329e768f94b56a32957dc17e7d72d027bed84ba947f026ba1c068b27d70a371" gracePeriod=30
Oct 30 22:15:31.160209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:31.151770    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"]
Oct 30 22:15:31.160209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:31.152106    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt" podUID="de56be49-5476-40e9-a737-0b12ccfdab60" containerName="istio-proxy" containerID="cri-o://c73a7f805b41b55c5fa63bb3ec0f3ea5441153638520d494dcf40854e773140c" gracePeriod=30
Oct 30 22:15:31.166045 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:31.166017    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"]
Oct 30 22:15:31.166243 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:31.166200    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk" podUID="5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" containerName="istio-proxy" containerID="cri-o://2a58a13217a584025c66ea52f5f23d532d337f6723918157a0776c7e0ad24eb5" gracePeriod=30
Oct 30 22:15:31.224139 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:31.224096    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:32.039588 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:32.039553    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:32.258694 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:32.258655    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:32.903895 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:32.903866    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:33.364243 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:33.364149    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:15:33.364243 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:33.364218    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt" podUID="de56be49-5476-40e9-a737-0b12ccfdab60" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:15:33.509069 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:33.509035    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:15:33.509281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:33.509089    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk" podUID="5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:15:34.235723 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:34.235681    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:15:34.235908 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:34.235752    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc" podUID="dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:15:36.241598 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.241556    5586 patch_prober.go:29] interesting pod/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:15:36.242034 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.241620    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Oct 30 22:15:36.438156 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.438121    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:15:36.498518 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.498489    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:15:36.501750 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.501728    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:15:36.505452 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.505435    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:15:36.533691 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.533664    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-workload-socket\") pod \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") "
Oct 30 22:15:36.533814 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.533701    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istiod-ca-cert\") pod \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") "
Oct 30 22:15:36.533814 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.533721    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-credential-socket\") pod \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") "
Oct 30 22:15:36.533814 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.533747    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-credential-socket\") pod \"de56be49-5476-40e9-a737-0b12ccfdab60\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") "
Oct 30 22:15:36.533814 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.533801    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-workload-socket\") pod \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") "
Oct 30 22:15:36.534025 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.533837    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a89955b-47d7-4a3a-895f-6c3f69b695aa-kube-api-access-bv94v\" (UniqueName: \"kubernetes.io/projected/8a89955b-47d7-4a3a-895f-6c3f69b695aa-kube-api-access-bv94v\") pod \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") "
Oct 30 22:15:36.534025 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.533865    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-data\") pod \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") "
Oct 30 22:15:36.534190 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.533957    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-workload-socket" pod "8a89955b-47d7-4a3a-895f-6c3f69b695aa" (UID: "8a89955b-47d7-4a3a-895f-6c3f69b695aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.534190 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534180    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-token\" (UniqueName: \"kubernetes.io/projected/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-token\") pod \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") "
Oct 30 22:15:36.534306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.533974    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-credential-socket" pod "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" (UID: "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.534306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534096    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-data" pod "8a89955b-47d7-4a3a-895f-6c3f69b695aa" (UID: "8a89955b-47d7-4a3a-895f-6c3f69b695aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.534306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534135    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-workload-socket" pod "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" (UID: "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.534306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534092    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-credential-socket" pod "de56be49-5476-40e9-a737-0b12ccfdab60" (UID: "de56be49-5476-40e9-a737-0b12ccfdab60"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.534306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534166    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istiod-ca-cert" pod "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" (UID: "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:36.534306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534222    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istiod-ca-cert\") pod \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") "
Oct 30 22:15:36.534306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534295    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de56be49-5476-40e9-a737-0b12ccfdab60-istio-token\" (UniqueName: \"kubernetes.io/projected/de56be49-5476-40e9-a737-0b12ccfdab60-istio-token\") pod \"de56be49-5476-40e9-a737-0b12ccfdab60\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") "
Oct 30 22:15:36.534609 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534316    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-podinfo\") pod \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") "
Oct 30 22:15:36.534609 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534345    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-credential-socket\") pod \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") "
Oct 30 22:15:36.534609 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534370    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-envoy\") pod \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") "
Oct 30 22:15:36.534609 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534548    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istiod-ca-cert" pod "8a89955b-47d7-4a3a-895f-6c3f69b695aa" (UID: "8a89955b-47d7-4a3a-895f-6c3f69b695aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:36.534854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534641    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-credential-socket" pod "8a89955b-47d7-4a3a-895f-6c3f69b695aa" (UID: "8a89955b-47d7-4a3a-895f-6c3f69b695aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.534854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534553    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-data\") pod \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") "
Oct 30 22:15:36.534854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534734    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-workload-certs\") pod \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") "
Oct 30 22:15:36.534854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534768    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/de56be49-5476-40e9-a737-0b12ccfdab60-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de56be49-5476-40e9-a737-0b12ccfdab60-istio-podinfo\") pod \"de56be49-5476-40e9-a737-0b12ccfdab60\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") "
Oct 30 22:15:36.534854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534789    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-data" pod "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" (UID: "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.534854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534797    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-podinfo\") pod \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") "
Oct 30 22:15:36.534854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534830    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-podinfo\") pod \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") "
Oct 30 22:15:36.535220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534870    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de56be49-5476-40e9-a737-0b12ccfdab60-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de56be49-5476-40e9-a737-0b12ccfdab60-istiod-ca-cert\") pod \"de56be49-5476-40e9-a737-0b12ccfdab60\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") "
Oct 30 22:15:36.535220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534897    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-workload-socket\") pod \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") "
Oct 30 22:15:36.535220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.534935    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-envoy\") pod \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.535490    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-workload-certs\") pod \"de56be49-5476-40e9-a737-0b12ccfdab60\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.535552    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-token\" (UniqueName: \"kubernetes.io/projected/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-token\") pod \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.535597    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-token\" (UniqueName: \"kubernetes.io/projected/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-token\") pod \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.535645    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-data\") pod \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.535667    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-workload-socket" pod "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" (UID: "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.535683    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-envoy\") pod \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.535715    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-istio-data\") pod \"de56be49-5476-40e9-a737-0b12ccfdab60\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.535784    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-kube-api-access-ppzbd\" (UniqueName: \"kubernetes.io/projected/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-kube-api-access-ppzbd\") pod \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.535837    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-istio-envoy\") pod \"de56be49-5476-40e9-a737-0b12ccfdab60\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.535906    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-workload-certs\") pod \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\" (UID: \"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.535941    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-workload-socket\") pod \"de56be49-5476-40e9-a737-0b12ccfdab60\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536000    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de56be49-5476-40e9-a737-0b12ccfdab60-kube-api-access-cjsjq\" (UniqueName: \"kubernetes.io/projected/de56be49-5476-40e9-a737-0b12ccfdab60-kube-api-access-cjsjq\") pod \"de56be49-5476-40e9-a737-0b12ccfdab60\" (UID: \"de56be49-5476-40e9-a737-0b12ccfdab60\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536031    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-credential-socket\") pod \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536061    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-workload-certs\") pod \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\" (UID: \"8a89955b-47d7-4a3a-895f-6c3f69b695aa\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536096    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-kube-api-access-v82hh\" (UniqueName: \"kubernetes.io/projected/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-kube-api-access-v82hh\") pod \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536128    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istiod-ca-cert\") pod \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\" (UID: \"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5\") "
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536773    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536795    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536810    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536823    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536839    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536854    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536868    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536884    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536898    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536914    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.535712    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de56be49-5476-40e9-a737-0b12ccfdab60-istiod-ca-cert" pod "de56be49-5476-40e9-a737-0b12ccfdab60" (UID: "de56be49-5476-40e9-a737-0b12ccfdab60"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536051    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-istio-data" pod "de56be49-5476-40e9-a737-0b12ccfdab60" (UID: "de56be49-5476-40e9-a737-0b12ccfdab60"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536329    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-data" pod "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" (UID: "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536556    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-workload-certs" pod "de56be49-5476-40e9-a737-0b12ccfdab60" (UID: "de56be49-5476-40e9-a737-0b12ccfdab60"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.536769    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-workload-certs" pod "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" (UID: "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.537006    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-credential-socket" pod "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" (UID: "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.537359    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-workload-certs" pod "8a89955b-47d7-4a3a-895f-6c3f69b695aa" (UID: "8a89955b-47d7-4a3a-895f-6c3f69b695aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.537491    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-token" pod "8a89955b-47d7-4a3a-895f-6c3f69b695aa" (UID: "8a89955b-47d7-4a3a-895f-6c3f69b695aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.537678    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istiod-ca-cert" pod "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" (UID: "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:36.537877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.537710    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-workload-certs" pod "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" (UID: "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.539487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.538113    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a89955b-47d7-4a3a-895f-6c3f69b695aa-kube-api-access-bv94v" pod "8a89955b-47d7-4a3a-895f-6c3f69b695aa" (UID: "8a89955b-47d7-4a3a-895f-6c3f69b695aa"). InnerVolumeSpecName "kube-api-access-bv94v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:36.539487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.538414    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de56be49-5476-40e9-a737-0b12ccfdab60-istio-token" pod "de56be49-5476-40e9-a737-0b12ccfdab60" (UID: "de56be49-5476-40e9-a737-0b12ccfdab60"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:36.539571 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.539515    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-envoy" pod "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" (UID: "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.539680 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.539656    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-workload-socket" pod "de56be49-5476-40e9-a737-0b12ccfdab60" (UID: "de56be49-5476-40e9-a737-0b12ccfdab60"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.539935 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.539912    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-token" pod "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" (UID: "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:36.540025 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.539948    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-podinfo" pod "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" (UID: "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:15:36.540084 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.540070    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-podinfo" pod "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" (UID: "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:15:36.540139 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.540107    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-envoy" pod "8a89955b-47d7-4a3a-895f-6c3f69b695aa" (UID: "8a89955b-47d7-4a3a-895f-6c3f69b695aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.540270 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.540183    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-podinfo" pod "8a89955b-47d7-4a3a-895f-6c3f69b695aa" (UID: "8a89955b-47d7-4a3a-895f-6c3f69b695aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:15:36.540543 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.540522    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-envoy" pod "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" (UID: "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.540636 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.540550    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-istio-envoy" pod "de56be49-5476-40e9-a737-0b12ccfdab60" (UID: "de56be49-5476-40e9-a737-0b12ccfdab60"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:36.540636 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.540565    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de56be49-5476-40e9-a737-0b12ccfdab60-kube-api-access-cjsjq" pod "de56be49-5476-40e9-a737-0b12ccfdab60" (UID: "de56be49-5476-40e9-a737-0b12ccfdab60"). InnerVolumeSpecName "kube-api-access-cjsjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:36.540929 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.540904    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-token" pod "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" (UID: "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:36.541037 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.540971    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-kube-api-access-ppzbd" pod "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" (UID: "dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf"). InnerVolumeSpecName "kube-api-access-ppzbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:36.541037 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.541018    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-kube-api-access-v82hh" pod "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" (UID: "5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5"). InnerVolumeSpecName "kube-api-access-v82hh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:36.541731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.541709    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/de56be49-5476-40e9-a737-0b12ccfdab60-istio-podinfo" pod "de56be49-5476-40e9-a737-0b12ccfdab60" (UID: "de56be49-5476-40e9-a737-0b12ccfdab60"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:15:36.638425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638347    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638374    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638383    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638393    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de56be49-5476-40e9-a737-0b12ccfdab60-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638402    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638411    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638420    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de56be49-5476-40e9-a737-0b12ccfdab60-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638430    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638440    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638448    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638456    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638464    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638472    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638481    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638490    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppzbd\" (UniqueName: \"kubernetes.io/projected/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-kube-api-access-ppzbd\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638499    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638507    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638516    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de56be49-5476-40e9-a737-0b12ccfdab60-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638525    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjsjq\" (UniqueName: \"kubernetes.io/projected/de56be49-5476-40e9-a737-0b12ccfdab60-kube-api-access-cjsjq\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638534    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638541    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a89955b-47d7-4a3a-895f-6c3f69b695aa-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638550    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v82hh\" (UniqueName: \"kubernetes.io/projected/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-kube-api-access-v82hh\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638558    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638566    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bv94v\" (UniqueName: \"kubernetes.io/projected/8a89955b-47d7-4a3a-895f-6c3f69b695aa-kube-api-access-bv94v\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638575    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a89955b-47d7-4a3a-895f-6c3f69b695aa-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:36.638768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:36.638583    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de56be49-5476-40e9-a737-0b12ccfdab60-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:37.056156 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.056119    5586 generic.go:356] "Generic (PLEG): container finished" podID="5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" containerID="2a58a13217a584025c66ea52f5f23d532d337f6723918157a0776c7e0ad24eb5" exitCode=0
Oct 30 22:15:37.056346 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.056169    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"
Oct 30 22:15:37.056346 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.056205    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk" event={"ID":"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5","Type":"ContainerDied","Data":"2a58a13217a584025c66ea52f5f23d532d337f6723918157a0776c7e0ad24eb5"}
Oct 30 22:15:37.056346 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.056274    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk" event={"ID":"5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5","Type":"ContainerDied","Data":"c115fcf06943f5627e95f073301e23495892a54df9ebc0cf6c3c40d432dfbf52"}
Oct 30 22:15:37.056346 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.056299    5586 scope.go:122] "RemoveContainer" containerID="2a58a13217a584025c66ea52f5f23d532d337f6723918157a0776c7e0ad24eb5"
Oct 30 22:15:37.058334 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.058309    5586 generic.go:356] "Generic (PLEG): container finished" podID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerID="881994a42458a3976ce76c6d80b1856d70db6638c4c2f4457c4fbf2ec417f995" exitCode=0
Oct 30 22:15:37.058472 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.058340    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" event={"ID":"8a89955b-47d7-4a3a-895f-6c3f69b695aa","Type":"ContainerDied","Data":"881994a42458a3976ce76c6d80b1856d70db6638c4c2f4457c4fbf2ec417f995"}
Oct 30 22:15:37.058472 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.058378    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"
Oct 30 22:15:37.058472 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.058387    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb" event={"ID":"8a89955b-47d7-4a3a-895f-6c3f69b695aa","Type":"ContainerDied","Data":"df37d5d10d84443ece8133aed532c63771332bba20ff286382f614671e97c07c"}
Oct 30 22:15:37.059972 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.059929    5586 generic.go:356] "Generic (PLEG): container finished" podID="dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" containerID="0329e768f94b56a32957dc17e7d72d027bed84ba947f026ba1c068b27d70a371" exitCode=0
Oct 30 22:15:37.059972 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.059957    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc" event={"ID":"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf","Type":"ContainerDied","Data":"0329e768f94b56a32957dc17e7d72d027bed84ba947f026ba1c068b27d70a371"}
Oct 30 22:15:37.060147 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.059983    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"
Oct 30 22:15:37.060147 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.060002    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc" event={"ID":"dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf","Type":"ContainerDied","Data":"b4a9ff65925e8076eeecf11cc25a9e12f9c39eb7d099809313b28c8a1baf4559"}
Oct 30 22:15:37.061589 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.061566    5586 generic.go:356] "Generic (PLEG): container finished" podID="de56be49-5476-40e9-a737-0b12ccfdab60" containerID="c73a7f805b41b55c5fa63bb3ec0f3ea5441153638520d494dcf40854e773140c" exitCode=0
Oct 30 22:15:37.061702 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.061606    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt" event={"ID":"de56be49-5476-40e9-a737-0b12ccfdab60","Type":"ContainerDied","Data":"c73a7f805b41b55c5fa63bb3ec0f3ea5441153638520d494dcf40854e773140c"}
Oct 30 22:15:37.061702 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.061610    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"
Oct 30 22:15:37.061702 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.061644    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt" event={"ID":"de56be49-5476-40e9-a737-0b12ccfdab60","Type":"ContainerDied","Data":"cf71747dae46bcef053321241488657f5164bb4a1a86967340977ea34f38ffe8"}
Oct 30 22:15:37.077197 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.077174    5586 scope.go:122] "RemoveContainer" containerID="2a58a13217a584025c66ea52f5f23d532d337f6723918157a0776c7e0ad24eb5"
Oct 30 22:15:37.077699 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:37.077666    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a58a13217a584025c66ea52f5f23d532d337f6723918157a0776c7e0ad24eb5\": container with ID starting with 2a58a13217a584025c66ea52f5f23d532d337f6723918157a0776c7e0ad24eb5 not found: ID does not exist" containerID="2a58a13217a584025c66ea52f5f23d532d337f6723918157a0776c7e0ad24eb5"
Oct 30 22:15:37.077794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.077709    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a58a13217a584025c66ea52f5f23d532d337f6723918157a0776c7e0ad24eb5"} err="failed to get container status \"2a58a13217a584025c66ea52f5f23d532d337f6723918157a0776c7e0ad24eb5\": rpc error: code = NotFound desc = could not find container \"2a58a13217a584025c66ea52f5f23d532d337f6723918157a0776c7e0ad24eb5\": container with ID starting with 2a58a13217a584025c66ea52f5f23d532d337f6723918157a0776c7e0ad24eb5 not found: ID does not exist"
Oct 30 22:15:37.077794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.077735    5586 scope.go:122] "RemoveContainer" containerID="881994a42458a3976ce76c6d80b1856d70db6638c4c2f4457c4fbf2ec417f995"
Oct 30 22:15:37.088401 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.088381    5586 scope.go:122] "RemoveContainer" containerID="d525a8b9d7cfa3cd8d201500ae8112c65218a8b7eb984ccb8324448ae8d0cdab"
Oct 30 22:15:37.096975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.096952    5586 scope.go:122] "RemoveContainer" containerID="881994a42458a3976ce76c6d80b1856d70db6638c4c2f4457c4fbf2ec417f995"
Oct 30 22:15:37.097268 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:37.097225    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"881994a42458a3976ce76c6d80b1856d70db6638c4c2f4457c4fbf2ec417f995\": container with ID starting with 881994a42458a3976ce76c6d80b1856d70db6638c4c2f4457c4fbf2ec417f995 not found: ID does not exist" containerID="881994a42458a3976ce76c6d80b1856d70db6638c4c2f4457c4fbf2ec417f995"
Oct 30 22:15:37.097333 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.097281    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"881994a42458a3976ce76c6d80b1856d70db6638c4c2f4457c4fbf2ec417f995"} err="failed to get container status \"881994a42458a3976ce76c6d80b1856d70db6638c4c2f4457c4fbf2ec417f995\": rpc error: code = NotFound desc = could not find container \"881994a42458a3976ce76c6d80b1856d70db6638c4c2f4457c4fbf2ec417f995\": container with ID starting with 881994a42458a3976ce76c6d80b1856d70db6638c4c2f4457c4fbf2ec417f995 not found: ID does not exist"
Oct 30 22:15:37.097333 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.097306    5586 scope.go:122] "RemoveContainer" containerID="d525a8b9d7cfa3cd8d201500ae8112c65218a8b7eb984ccb8324448ae8d0cdab"
Oct 30 22:15:37.097581 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:37.097562    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d525a8b9d7cfa3cd8d201500ae8112c65218a8b7eb984ccb8324448ae8d0cdab\": container with ID starting with d525a8b9d7cfa3cd8d201500ae8112c65218a8b7eb984ccb8324448ae8d0cdab not found: ID does not exist" containerID="d525a8b9d7cfa3cd8d201500ae8112c65218a8b7eb984ccb8324448ae8d0cdab"
Oct 30 22:15:37.097643 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.097589    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d525a8b9d7cfa3cd8d201500ae8112c65218a8b7eb984ccb8324448ae8d0cdab"} err="failed to get container status \"d525a8b9d7cfa3cd8d201500ae8112c65218a8b7eb984ccb8324448ae8d0cdab\": rpc error: code = NotFound desc = could not find container \"d525a8b9d7cfa3cd8d201500ae8112c65218a8b7eb984ccb8324448ae8d0cdab\": container with ID starting with d525a8b9d7cfa3cd8d201500ae8112c65218a8b7eb984ccb8324448ae8d0cdab not found: ID does not exist"
Oct 30 22:15:37.097643 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.097610    5586 scope.go:122] "RemoveContainer" containerID="0329e768f94b56a32957dc17e7d72d027bed84ba947f026ba1c068b27d70a371"
Oct 30 22:15:37.106019 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.106003    5586 scope.go:122] "RemoveContainer" containerID="0329e768f94b56a32957dc17e7d72d027bed84ba947f026ba1c068b27d70a371"
Oct 30 22:15:37.106288 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:37.106271    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0329e768f94b56a32957dc17e7d72d027bed84ba947f026ba1c068b27d70a371\": container with ID starting with 0329e768f94b56a32957dc17e7d72d027bed84ba947f026ba1c068b27d70a371 not found: ID does not exist" containerID="0329e768f94b56a32957dc17e7d72d027bed84ba947f026ba1c068b27d70a371"
Oct 30 22:15:37.106357 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.106298    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0329e768f94b56a32957dc17e7d72d027bed84ba947f026ba1c068b27d70a371"} err="failed to get container status \"0329e768f94b56a32957dc17e7d72d027bed84ba947f026ba1c068b27d70a371\": rpc error: code = NotFound desc = could not find container \"0329e768f94b56a32957dc17e7d72d027bed84ba947f026ba1c068b27d70a371\": container with ID starting with 0329e768f94b56a32957dc17e7d72d027bed84ba947f026ba1c068b27d70a371 not found: ID does not exist"
Oct 30 22:15:37.106357 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.106323    5586 scope.go:122] "RemoveContainer" containerID="c73a7f805b41b55c5fa63bb3ec0f3ea5441153638520d494dcf40854e773140c"
Oct 30 22:15:37.114760 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.114745    5586 scope.go:122] "RemoveContainer" containerID="c73a7f805b41b55c5fa63bb3ec0f3ea5441153638520d494dcf40854e773140c"
Oct 30 22:15:37.115019 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:37.115001    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c73a7f805b41b55c5fa63bb3ec0f3ea5441153638520d494dcf40854e773140c\": container with ID starting with c73a7f805b41b55c5fa63bb3ec0f3ea5441153638520d494dcf40854e773140c not found: ID does not exist" containerID="c73a7f805b41b55c5fa63bb3ec0f3ea5441153638520d494dcf40854e773140c"
Oct 30 22:15:37.115084 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.115025    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c73a7f805b41b55c5fa63bb3ec0f3ea5441153638520d494dcf40854e773140c"} err="failed to get container status \"c73a7f805b41b55c5fa63bb3ec0f3ea5441153638520d494dcf40854e773140c\": rpc error: code = NotFound desc = could not find container \"c73a7f805b41b55c5fa63bb3ec0f3ea5441153638520d494dcf40854e773140c\": container with ID starting with c73a7f805b41b55c5fa63bb3ec0f3ea5441153638520d494dcf40854e773140c not found: ID does not exist"
Oct 30 22:15:37.229652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.229613    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"]
Oct 30 22:15:37.240570 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.240537    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-88lbb"]
Oct 30 22:15:37.352364 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.352278    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"]
Oct 30 22:15:37.354911 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.354890    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-fthlt"]
Oct 30 22:15:37.447113 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.447081    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" path="/var/lib/kubelet/pods/8a89955b-47d7-4a3a-895f-6c3f69b695aa/volumes"
Oct 30 22:15:37.447609 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.447595    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de56be49-5476-40e9-a737-0b12ccfdab60" path="/var/lib/kubelet/pods/de56be49-5476-40e9-a737-0b12ccfdab60/volumes"
Oct 30 22:15:37.460850 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.460819    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"]
Oct 30 22:15:37.467042 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.467017    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-zvjmc"]
Oct 30 22:15:37.578616 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.578585    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"]
Oct 30 22:15:37.585545 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.585509    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6df3ba0e-openshift-default-5fdd5fcfbf-c4tzk"]
Oct 30 22:15:37.701280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:37.701212    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:39.100906 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.100877    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"]
Oct 30 22:15:39.101402 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101388    5586 cpu_manager.go:414] "containerMap: removing container" podUID="7ad8b80f-746c-4450-a732-3551846186c7" containerName="cleanup"
Oct 30 22:15:39.101443 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101402    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ad8b80f-746c-4450-a732-3551846186c7" containerName="cleanup"
Oct 30 22:15:39.101443 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101416    5586 cpu_manager.go:414] "containerMap: removing container" podUID="dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" containerName="istio-proxy"
Oct 30 22:15:39.101443 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101422    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" containerName="istio-proxy"
Oct 30 22:15:39.101443 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101433    5586 cpu_manager.go:414] "containerMap: removing container" podUID="07444e4a-f484-4ff4-bffd-9c3f329fca00" containerName="cleanup"
Oct 30 22:15:39.101443 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101438    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07444e4a-f484-4ff4-bffd-9c3f329fca00" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101446    5586 cpu_manager.go:414] "containerMap: removing container" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101450    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101460    5586 cpu_manager.go:414] "containerMap: removing container" podUID="003da65f-5b3a-4735-bf4d-8f064ad48d96" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101464    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="003da65f-5b3a-4735-bf4d-8f064ad48d96" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101473    5586 cpu_manager.go:414] "containerMap: removing container" podUID="6e70ec99-2141-4616-b598-84675558b246" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101478    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e70ec99-2141-4616-b598-84675558b246" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101486    5586 cpu_manager.go:414] "containerMap: removing container" podUID="de56be49-5476-40e9-a737-0b12ccfdab60" containerName="istio-proxy"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101491    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de56be49-5476-40e9-a737-0b12ccfdab60" containerName="istio-proxy"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101497    5586 cpu_manager.go:414] "containerMap: removing container" podUID="5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" containerName="istio-proxy"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101502    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" containerName="istio-proxy"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101510    5586 cpu_manager.go:414] "containerMap: removing container" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101514    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101521    5586 cpu_manager.go:414] "containerMap: removing container" podUID="6e70ec99-2141-4616-b598-84675558b246" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101525    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e70ec99-2141-4616-b598-84675558b246" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101532    5586 cpu_manager.go:414] "containerMap: removing container" podUID="37583645-5073-494b-8079-7eac9ceb5d30" containerName="istio-proxy"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101537    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37583645-5073-494b-8079-7eac9ceb5d30" containerName="istio-proxy"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101552    5586 cpu_manager.go:414] "containerMap: removing container" podUID="003da65f-5b3a-4735-bf4d-8f064ad48d96" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101557    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="003da65f-5b3a-4735-bf4d-8f064ad48d96" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101564    5586 cpu_manager.go:414] "containerMap: removing container" podUID="7ad8b80f-746c-4450-a732-3551846186c7" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101569    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ad8b80f-746c-4450-a732-3551846186c7" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101576    5586 cpu_manager.go:414] "containerMap: removing container" podUID="6ad3229f-17cf-4b4a-8cdc-459636c6ff7e" containerName="revoke-keys"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101581    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ad3229f-17cf-4b4a-8cdc-459636c6ff7e" containerName="revoke-keys"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101589    5586 cpu_manager.go:414] "containerMap: removing container" podUID="7ad8b80f-746c-4450-a732-3551846186c7" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101595    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ad8b80f-746c-4450-a732-3551846186c7" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101609    5586 cpu_manager.go:414] "containerMap: removing container" podUID="003da65f-5b3a-4735-bf4d-8f064ad48d96" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101614    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="003da65f-5b3a-4735-bf4d-8f064ad48d96" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101623    5586 cpu_manager.go:414] "containerMap: removing container" podUID="07444e4a-f484-4ff4-bffd-9c3f329fca00" containerName="cleanup"
Oct 30 22:15:39.101628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101627    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07444e4a-f484-4ff4-bffd-9c3f329fca00" containerName="cleanup"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101714    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e70ec99-2141-4616-b598-84675558b246" containerName="cleanup"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101723    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="de56be49-5476-40e9-a737-0b12ccfdab60" containerName="istio-proxy"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101732    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" containerName="istio-proxy"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101738    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101744    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e70ec99-2141-4616-b598-84675558b246" containerName="cleanup"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101751    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="003da65f-5b3a-4735-bf4d-8f064ad48d96" containerName="cleanup"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101756    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ad8b80f-746c-4450-a732-3551846186c7" containerName="cleanup"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101762    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ad3229f-17cf-4b4a-8cdc-459636c6ff7e" containerName="revoke-keys"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101767    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ad8b80f-746c-4450-a732-3551846186c7" containerName="cleanup"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101773    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="07444e4a-f484-4ff4-bffd-9c3f329fca00" containerName="cleanup"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101778    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="003da65f-5b3a-4735-bf4d-8f064ad48d96" containerName="cleanup"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101783    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="07444e4a-f484-4ff4-bffd-9c3f329fca00" containerName="cleanup"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101788    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ad8b80f-746c-4450-a732-3551846186c7" containerName="cleanup"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101793    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" containerName="istio-proxy"
Oct 30 22:15:39.102469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.101798    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="07444e4a-f484-4ff4-bffd-9c3f329fca00" containerName="cleanup"
Oct 30 22:15:39.107134 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.107113    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.159458 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.159426    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-c79cb5-openshift-default-dockercfg-7289d\""
Oct 30 22:15:39.167506 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.167478    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh6kg\" (UniqueName: \"kubernetes.io/projected/7db6ba06-6961-482a-8453-8ee12cbbb85c-kube-api-access-hh6kg\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.167635 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.167521    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7db6ba06-6961-482a-8453-8ee12cbbb85c-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.167635 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.167545    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.167717 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.167694    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.167790 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.167728    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.167849 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.167830    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.167912 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.167892    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.167971 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.167940    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.168061 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.168046    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.213100 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.213068    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"]
Oct 30 22:15:39.268731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.268697    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.268911 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.268754    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh6kg\" (UniqueName: \"kubernetes.io/projected/7db6ba06-6961-482a-8453-8ee12cbbb85c-kube-api-access-hh6kg\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.268911 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.268800    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7db6ba06-6961-482a-8453-8ee12cbbb85c-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.268911 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.268829    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.268911 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.268880    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.269113 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.268918    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.269113 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.268972    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.269113 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.269015    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.269113 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.269050    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.269510 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.269478    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-workload-certs\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.269655 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.269567    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-data\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.269739 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.269691    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-workload-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.269739 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.269723    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7db6ba06-6961-482a-8453-8ee12cbbb85c-istiod-ca-cert\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.269930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.269858    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-credential-socket\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.272018 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.271991    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-podinfo\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.272324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.272302    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-envoy\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.325905 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.325852    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-token\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.326102 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.325905    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh6kg\" (UniqueName: \"kubernetes.io/projected/7db6ba06-6961-482a-8453-8ee12cbbb85c-kube-api-access-hh6kg\") pod \"e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") " pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.421420 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.421381    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:39.449446 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.449411    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5" path="/var/lib/kubelet/pods/5c85fcdc-ee12-4b9b-8d38-ad82c0e398a5/volumes"
Oct 30 22:15:39.450041 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.450018    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf" path="/var/lib/kubelet/pods/dc6b9f2d-596c-4c77-ae0a-ae38b52a64bf/volumes"
Oct 30 22:15:39.881526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.881496    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"]
Oct 30 22:15:39.882934 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:15:39.882901    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db6ba06_6961_482a_8453_8ee12cbbb85c.slice/crio-0f2c154077062132514526b6f3063acf9f9ed774e94e53583ce5a8065d29e5f3 WatchSource:0}: Error finding container 0f2c154077062132514526b6f3063acf9f9ed774e94e53583ce5a8065d29e5f3: Status 404 returned error can't find the container with id 0f2c154077062132514526b6f3063acf9f9ed774e94e53583ce5a8065d29e5f3
Oct 30 22:15:39.885226 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.885194    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:15:39.885307 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.885291    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:15:39.885360 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:39.885349    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:15:40.090365 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:40.090330    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp" event={"ID":"7db6ba06-6961-482a-8453-8ee12cbbb85c","Type":"ContainerStarted","Data":"4b01ac5d73f44c5e414e07d94c8667f2a592cc535178371599cfed78fe13ae8a"}
Oct 30 22:15:40.090581 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:40.090396    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp" event={"ID":"7db6ba06-6961-482a-8453-8ee12cbbb85c","Type":"ContainerStarted","Data":"0f2c154077062132514526b6f3063acf9f9ed774e94e53583ce5a8065d29e5f3"}
Oct 30 22:15:40.210307 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:40.210251    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp" podStartSLOduration=1.210220384 podStartE2EDuration="1.210220384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:40.207437302 +0000 UTC m=+2482.879341239" watchObservedRunningTime="2026-10-30 22:15:40.210220384 +0000 UTC m=+2482.882124319"
Oct 30 22:15:40.422265 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:40.422200    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:40.423827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:40.423799    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.153:15021/healthz/ready\": dial tcp 10.134.0.153:15021: connect: connection refused" start-of-body=
Oct 30 22:15:40.423960 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:40.423856    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp" podUID="7db6ba06-6961-482a-8453-8ee12cbbb85c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.153:15021/healthz/ready\": dial tcp 10.134.0.153:15021: connect: connection refused"
Oct 30 22:15:40.432758 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:40.432728    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:41.430256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:41.425678    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.153:15021/healthz/ready\": dial tcp 10.134.0.153:15021: connect: connection refused" start-of-body=
Oct 30 22:15:41.430256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:41.425748    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp" podUID="7db6ba06-6961-482a-8453-8ee12cbbb85c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.153:15021/healthz/ready\": dial tcp 10.134.0.153:15021: connect: connection refused"
Oct 30 22:15:42.432259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:42.425717    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.153:15021/healthz/ready\": dial tcp 10.134.0.153:15021: connect: connection refused" start-of-body=
Oct 30 22:15:42.432259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:42.425787    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp" podUID="7db6ba06-6961-482a-8453-8ee12cbbb85c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.153:15021/healthz/ready\": dial tcp 10.134.0.153:15021: connect: connection refused"
Oct 30 22:15:42.527265 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:42.526802    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:43.439257 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:43.438688    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.153:15021/healthz/ready\": dial tcp 10.134.0.153:15021: connect: connection refused" start-of-body=
Oct 30 22:15:43.439257 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:43.438759    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp" podUID="7db6ba06-6961-482a-8453-8ee12cbbb85c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.153:15021/healthz/ready\": dial tcp 10.134.0.153:15021: connect: connection refused"
Oct 30 22:15:44.422819 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:44.422771    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.153:15021/healthz/ready\": dial tcp 10.134.0.153:15021: connect: connection refused" start-of-body=
Oct 30 22:15:44.423191 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:44.423156    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp" podUID="7db6ba06-6961-482a-8453-8ee12cbbb85c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.153:15021/healthz/ready\": dial tcp 10.134.0.153:15021: connect: connection refused"
Oct 30 22:15:45.455259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:45.433786    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.153:15021/healthz/ready\": dial tcp 10.134.0.153:15021: connect: connection refused" start-of-body=
Oct 30 22:15:45.455259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:45.433924    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp" podUID="7db6ba06-6961-482a-8453-8ee12cbbb85c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.153:15021/healthz/ready\": dial tcp 10.134.0.153:15021: connect: connection refused"
Oct 30 22:15:46.258432 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:46.258380    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:46.424769 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:46.424722    5586 patch_prober.go:29] interesting pod/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.153:15021/healthz/ready\": dial tcp 10.134.0.153:15021: connect: connection refused" start-of-body=
Oct 30 22:15:46.425075 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:46.425042    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp" podUID="7db6ba06-6961-482a-8453-8ee12cbbb85c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.153:15021/healthz/ready\": dial tcp 10.134.0.153:15021: connect: connection refused"
Oct 30 22:15:46.582323 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:46.581942    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:46.887356 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:46.882853    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:47.444674 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:47.444634    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:47.447335 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:47.447303    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:47.488553 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:47.483162    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:15:47.501769 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:47.501724    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:48.363864 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:15:48.363823    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:15:52.373187 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:52.373134    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:56.571691 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:56.571652    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:58.211487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:58.211436    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:58.369308 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:58.369263    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:58.530278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:58.530161    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:15:59.075015 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:15:59.074972    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:03.498622 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:03.498579    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:04.896078 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:04.896037    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:05.053980 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:05.053942    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:05.238249 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:05.238206    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"]
Oct 30 22:16:05.238494 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:05.238461    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp" podUID="7db6ba06-6961-482a-8453-8ee12cbbb85c" containerName="istio-proxy" containerID="cri-o://4b01ac5d73f44c5e414e07d94c8667f2a592cc535178371599cfed78fe13ae8a" gracePeriod=30
Oct 30 22:16:05.274487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:05.274447    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:08.182443 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:08.182406    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:10.423713 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.423675    5586 generic.go:356] "Generic (PLEG): container finished" podID="7db6ba06-6961-482a-8453-8ee12cbbb85c" containerID="4b01ac5d73f44c5e414e07d94c8667f2a592cc535178371599cfed78fe13ae8a" exitCode=0
Oct 30 22:16:10.424129 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.423724    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp" event={"ID":"7db6ba06-6961-482a-8453-8ee12cbbb85c","Type":"ContainerDied","Data":"4b01ac5d73f44c5e414e07d94c8667f2a592cc535178371599cfed78fe13ae8a"}
Oct 30 22:16:10.505741 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.505672    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:16:10.644947 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.644905    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-workload-certs\") pod \"7db6ba06-6961-482a-8453-8ee12cbbb85c\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") "
Oct 30 22:16:10.645157 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.644963    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-token\" (UniqueName: \"kubernetes.io/projected/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-token\") pod \"7db6ba06-6961-482a-8453-8ee12cbbb85c\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") "
Oct 30 22:16:10.645157 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.645008    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7db6ba06-6961-482a-8453-8ee12cbbb85c-kube-api-access-hh6kg\" (UniqueName: \"kubernetes.io/projected/7db6ba06-6961-482a-8453-8ee12cbbb85c-kube-api-access-hh6kg\") pod \"7db6ba06-6961-482a-8453-8ee12cbbb85c\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") "
Oct 30 22:16:10.645157 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.645052    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-credential-socket\") pod \"7db6ba06-6961-482a-8453-8ee12cbbb85c\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") "
Oct 30 22:16:10.645157 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.645144    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-data\") pod \"7db6ba06-6961-482a-8453-8ee12cbbb85c\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") "
Oct 30 22:16:10.645429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.645192    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-podinfo\") pod \"7db6ba06-6961-482a-8453-8ee12cbbb85c\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") "
Oct 30 22:16:10.645429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.645272    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-envoy\") pod \"7db6ba06-6961-482a-8453-8ee12cbbb85c\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") "
Oct 30 22:16:10.645429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.645293    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-workload-certs" pod "7db6ba06-6961-482a-8453-8ee12cbbb85c" (UID: "7db6ba06-6961-482a-8453-8ee12cbbb85c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.645429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.645346    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-workload-socket\") pod \"7db6ba06-6961-482a-8453-8ee12cbbb85c\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") "
Oct 30 22:16:10.645429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.645412    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7db6ba06-6961-482a-8453-8ee12cbbb85c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7db6ba06-6961-482a-8453-8ee12cbbb85c-istiod-ca-cert\") pod \"7db6ba06-6961-482a-8453-8ee12cbbb85c\" (UID: \"7db6ba06-6961-482a-8453-8ee12cbbb85c\") "
Oct 30 22:16:10.645429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.645414    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-credential-socket" pod "7db6ba06-6961-482a-8453-8ee12cbbb85c" (UID: "7db6ba06-6961-482a-8453-8ee12cbbb85c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.645757 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.645455    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-data" pod "7db6ba06-6961-482a-8453-8ee12cbbb85c" (UID: "7db6ba06-6961-482a-8453-8ee12cbbb85c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.645757 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.645610    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-workload-socket" pod "7db6ba06-6961-482a-8453-8ee12cbbb85c" (UID: "7db6ba06-6961-482a-8453-8ee12cbbb85c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.645862 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.645764    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7db6ba06-6961-482a-8453-8ee12cbbb85c-istiod-ca-cert" pod "7db6ba06-6961-482a-8453-8ee12cbbb85c" (UID: "7db6ba06-6961-482a-8453-8ee12cbbb85c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:10.646056 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.646028    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.646056 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.646058    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.646221 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.646072    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.646221 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.646086    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7db6ba06-6961-482a-8453-8ee12cbbb85c-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.646221 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.646098    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.647568 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.647536    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-podinfo" pod "7db6ba06-6961-482a-8453-8ee12cbbb85c" (UID: "7db6ba06-6961-482a-8453-8ee12cbbb85c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:16:10.647653 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.647629    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-envoy" pod "7db6ba06-6961-482a-8453-8ee12cbbb85c" (UID: "7db6ba06-6961-482a-8453-8ee12cbbb85c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:10.647863 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.647845    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7db6ba06-6961-482a-8453-8ee12cbbb85c-kube-api-access-hh6kg" pod "7db6ba06-6961-482a-8453-8ee12cbbb85c" (UID: "7db6ba06-6961-482a-8453-8ee12cbbb85c"). InnerVolumeSpecName "kube-api-access-hh6kg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:10.647926 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.647870    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-token" pod "7db6ba06-6961-482a-8453-8ee12cbbb85c" (UID: "7db6ba06-6961-482a-8453-8ee12cbbb85c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:10.747483 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.747444    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.747483 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.747479    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.747483 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.747492    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7db6ba06-6961-482a-8453-8ee12cbbb85c-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:10.747727 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:10.747504    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh6kg\" (UniqueName: \"kubernetes.io/projected/7db6ba06-6961-482a-8453-8ee12cbbb85c-kube-api-access-hh6kg\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:11.433041 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:11.433007    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"
Oct 30 22:16:11.433526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:11.433048    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp" event={"ID":"7db6ba06-6961-482a-8453-8ee12cbbb85c","Type":"ContainerDied","Data":"0f2c154077062132514526b6f3063acf9f9ed774e94e53583ce5a8065d29e5f3"}
Oct 30 22:16:11.433526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:11.433113    5586 scope.go:122] "RemoveContainer" containerID="4b01ac5d73f44c5e414e07d94c8667f2a592cc535178371599cfed78fe13ae8a"
Oct 30 22:16:11.599271 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:11.599197    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"]
Oct 30 22:16:11.602531 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:11.602502    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c79cb5-openshift-default-5464c54587-pd6rp"]
Oct 30 22:16:13.075162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:13.075121    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:13.449103 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:13.449060    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7db6ba06-6961-482a-8453-8ee12cbbb85c" path="/var/lib/kubelet/pods/7db6ba06-6961-482a-8453-8ee12cbbb85c/volumes"
Oct 30 22:16:14.390559 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:14.390523    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:14.560694 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:14.560653    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:17.875196 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:17.875156    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:18.362201 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:16:18.362165    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:16:19.714855 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:19.714818    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:22.971941 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:22.971902    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:24.664266 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:24.664195    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"]
Oct 30 22:16:24.713421 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:24.713387    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"]
Oct 30 22:16:24.828405 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:24.828346    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:27.785823 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:27.785784    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:30.209154 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:30.209064    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:30.560913 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:30.560823    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"]
Oct 30 22:16:30.663773 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:30.663735    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"]
Oct 30 22:16:30.940947 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:30.940904    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:31.135045 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:31.135012    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:32.572588 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:32.572543    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:35.562081 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:35.562033    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g" podUID="ede829db-672f-49fa-9b86-00d130c21560" containerName="payload-processing" containerID="cri-o://98dd6319631f816a0f877ad3a18a563734979b0ab0dadee09dd0b36ba8d08fd0" gracePeriod=25
Oct 30 22:16:35.664388 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:35.664345    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48" podUID="1fe6d48f-81e8-4900-8249-0a73a66fc86b" containerName="payload-pre-processing" containerID="cri-o://efc2e975250cfaedd0ce1f11668076a2d16f9e1a29a18b20cf8fee9169242e13" gracePeriod=25
Oct 30 22:16:36.257693 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:36.257651    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48" podUID="1fe6d48f-81e8-4900-8249-0a73a66fc86b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:16:36.257880 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:36.257667    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g" podUID="ede829db-672f-49fa-9b86-00d130c21560" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:16:37.365221 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:37.365189    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:39.685270 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:39.685209    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main" containerID="cri-o://5d2b4999d29de02d8d493c74e5d5833c4405d5cb78963897853aa4a94bbdac9c" gracePeriod=45
Oct 30 22:16:39.734294 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:39.734254    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" containerID="cri-o://662c4cf2dc22586d03f2a204406b8c712e1090ca4a0403b402301b39d90d0448" gracePeriod=45
Oct 30 22:16:39.830991 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:39.830954    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.61:8000/ready\": dial tcp 10.134.0.61:8000: connect: connection refused"
Oct 30 22:16:40.008787 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.008757    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 22:16:40.027919 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.027890    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 22:16:40.092389 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.092352    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-model-cache\") pod \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") "
Oct 30 22:16:40.092598 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.092416    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-dshm\") pod \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") "
Oct 30 22:16:40.092598 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.092445    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-tls-certs\" (UniqueName: \"kubernetes.io/secret/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-tls-certs\") pod \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") "
Oct 30 22:16:40.092598 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.092490    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-kube-api-access-k9gct\" (UniqueName: \"kubernetes.io/projected/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-kube-api-access-k9gct\") pod \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") "
Oct 30 22:16:40.092598 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.092597    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-tmp-dir\") pod \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") "
Oct 30 22:16:40.092831 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.092647    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-model-cache" pod "a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" (UID: "a63b784e-dd7f-4c97-b1f2-40644ef3d5e6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:40.092831 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.092661    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-home\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-home\") pod \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\" (UID: \"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6\") "
Oct 30 22:16:40.092926 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.092864    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-tmp-dir" pod "a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" (UID: "a63b784e-dd7f-4c97-b1f2-40644ef3d5e6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:40.092988 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.092950    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-home" pod "a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" (UID: "a63b784e-dd7f-4c97-b1f2-40644ef3d5e6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:40.093087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.093073    5586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-home\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:40.093158 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.093090    5586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-model-cache\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:40.093158 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.093101    5586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-tmp-dir\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:40.094853 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.094827    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-kube-api-access-k9gct" pod "a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" (UID: "a63b784e-dd7f-4c97-b1f2-40644ef3d5e6"). InnerVolumeSpecName "kube-api-access-k9gct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:40.095085 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.095063    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-dshm" pod "a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" (UID: "a63b784e-dd7f-4c97-b1f2-40644ef3d5e6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:40.095136 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.095067    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-tls-certs" pod "a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" (UID: "a63b784e-dd7f-4c97-b1f2-40644ef3d5e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:16:40.193803 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.193764    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e920b103-cdeb-4a9a-ac7f-447d89d5033f-kube-api-access-b5nbs\" (UniqueName: \"kubernetes.io/projected/e920b103-cdeb-4a9a-ac7f-447d89d5033f-kube-api-access-b5nbs\") pod \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") "
Oct 30 22:16:40.194014 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.193837    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-tmp-dir\") pod \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") "
Oct 30 22:16:40.194014 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.193997    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e920b103-cdeb-4a9a-ac7f-447d89d5033f-tls-certs\" (UniqueName: \"kubernetes.io/secret/e920b103-cdeb-4a9a-ac7f-447d89d5033f-tls-certs\") pod \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") "
Oct 30 22:16:40.194147 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.194070    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-model-cache\") pod \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") "
Oct 30 22:16:40.194147 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.194118    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-dshm\") pod \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") "
Oct 30 22:16:40.194227 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.194167    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-home\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-home\") pod \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\" (UID: \"e920b103-cdeb-4a9a-ac7f-447d89d5033f\") "
Oct 30 22:16:40.194581 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.194525    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-model-cache" pod "e920b103-cdeb-4a9a-ac7f-447d89d5033f" (UID: "e920b103-cdeb-4a9a-ac7f-447d89d5033f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:40.194581 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.194547    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9gct\" (UniqueName: \"kubernetes.io/projected/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-kube-api-access-k9gct\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:40.194581 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.194569    5586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-dshm\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:40.194581 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.194584    5586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6-tls-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:40.194962 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.194819    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-home" pod "e920b103-cdeb-4a9a-ac7f-447d89d5033f" (UID: "e920b103-cdeb-4a9a-ac7f-447d89d5033f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:40.194962 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.194913    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-tmp-dir" pod "e920b103-cdeb-4a9a-ac7f-447d89d5033f" (UID: "e920b103-cdeb-4a9a-ac7f-447d89d5033f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:40.196591 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.196568    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e920b103-cdeb-4a9a-ac7f-447d89d5033f-tls-certs" pod "e920b103-cdeb-4a9a-ac7f-447d89d5033f" (UID: "e920b103-cdeb-4a9a-ac7f-447d89d5033f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:16:40.196671 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.196609    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e920b103-cdeb-4a9a-ac7f-447d89d5033f-kube-api-access-b5nbs" pod "e920b103-cdeb-4a9a-ac7f-447d89d5033f" (UID: "e920b103-cdeb-4a9a-ac7f-447d89d5033f"). InnerVolumeSpecName "kube-api-access-b5nbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:40.197130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.197105    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-dshm" pod "e920b103-cdeb-4a9a-ac7f-447d89d5033f" (UID: "e920b103-cdeb-4a9a-ac7f-447d89d5033f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:40.295340 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.295221    5586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e920b103-cdeb-4a9a-ac7f-447d89d5033f-tls-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:40.295340 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.295287    5586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-model-cache\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:40.295340 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.295297    5586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-dshm\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:40.295340 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.295306    5586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-home\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:40.295340 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.295320    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5nbs\" (UniqueName: \"kubernetes.io/projected/e920b103-cdeb-4a9a-ac7f-447d89d5033f-kube-api-access-b5nbs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:40.295340 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.295329    5586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e920b103-cdeb-4a9a-ac7f-447d89d5033f-tmp-dir\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:40.630546 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.630449    5586 generic.go:356] "Generic (PLEG): container finished" podID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerID="662c4cf2dc22586d03f2a204406b8c712e1090ca4a0403b402301b39d90d0448" exitCode=0
Oct 30 22:16:40.630546 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.630501    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"
Oct 30 22:16:40.630812 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.630534    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" event={"ID":"e920b103-cdeb-4a9a-ac7f-447d89d5033f","Type":"ContainerDied","Data":"662c4cf2dc22586d03f2a204406b8c712e1090ca4a0403b402301b39d90d0448"}
Oct 30 22:16:40.630812 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.630579    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6" event={"ID":"e920b103-cdeb-4a9a-ac7f-447d89d5033f","Type":"ContainerDied","Data":"0a4089870469d01048069e5245b75121d267f47797916ef9c07bbb04bd045b58"}
Oct 30 22:16:40.630812 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.630602    5586 scope.go:122] "RemoveContainer" containerID="662c4cf2dc22586d03f2a204406b8c712e1090ca4a0403b402301b39d90d0448"
Oct 30 22:16:40.632038 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.632015    5586 generic.go:356] "Generic (PLEG): container finished" podID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerID="5d2b4999d29de02d8d493c74e5d5833c4405d5cb78963897853aa4a94bbdac9c" exitCode=0
Oct 30 22:16:40.632168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.632052    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" event={"ID":"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6","Type":"ContainerDied","Data":"5d2b4999d29de02d8d493c74e5d5833c4405d5cb78963897853aa4a94bbdac9c"}
Oct 30 22:16:40.632168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.632065    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"
Oct 30 22:16:40.632168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.632075    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2" event={"ID":"a63b784e-dd7f-4c97-b1f2-40644ef3d5e6","Type":"ContainerDied","Data":"cc93810ef710876237c6e74b680b5313f413d96addf4952e5ce39a9fc77fe5f0"}
Oct 30 22:16:40.641945 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.641922    5586 scope.go:122] "RemoveContainer" containerID="662c4cf2dc22586d03f2a204406b8c712e1090ca4a0403b402301b39d90d0448"
Oct 30 22:16:40.642338 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:16:40.642306    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"662c4cf2dc22586d03f2a204406b8c712e1090ca4a0403b402301b39d90d0448\": container with ID starting with 662c4cf2dc22586d03f2a204406b8c712e1090ca4a0403b402301b39d90d0448 not found: ID does not exist" containerID="662c4cf2dc22586d03f2a204406b8c712e1090ca4a0403b402301b39d90d0448"
Oct 30 22:16:40.642429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.642349    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"662c4cf2dc22586d03f2a204406b8c712e1090ca4a0403b402301b39d90d0448"} err="failed to get container status \"662c4cf2dc22586d03f2a204406b8c712e1090ca4a0403b402301b39d90d0448\": rpc error: code = NotFound desc = could not find container \"662c4cf2dc22586d03f2a204406b8c712e1090ca4a0403b402301b39d90d0448\": container with ID starting with 662c4cf2dc22586d03f2a204406b8c712e1090ca4a0403b402301b39d90d0448 not found: ID does not exist"
Oct 30 22:16:40.642429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.642377    5586 scope.go:122] "RemoveContainer" containerID="5d2b4999d29de02d8d493c74e5d5833c4405d5cb78963897853aa4a94bbdac9c"
Oct 30 22:16:40.651600 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.651582    5586 scope.go:122] "RemoveContainer" containerID="5d2b4999d29de02d8d493c74e5d5833c4405d5cb78963897853aa4a94bbdac9c"
Oct 30 22:16:40.651941 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:16:40.651917    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d2b4999d29de02d8d493c74e5d5833c4405d5cb78963897853aa4a94bbdac9c\": container with ID starting with 5d2b4999d29de02d8d493c74e5d5833c4405d5cb78963897853aa4a94bbdac9c not found: ID does not exist" containerID="5d2b4999d29de02d8d493c74e5d5833c4405d5cb78963897853aa4a94bbdac9c"
Oct 30 22:16:40.652017 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.651951    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d2b4999d29de02d8d493c74e5d5833c4405d5cb78963897853aa4a94bbdac9c"} err="failed to get container status \"5d2b4999d29de02d8d493c74e5d5833c4405d5cb78963897853aa4a94bbdac9c\": rpc error: code = NotFound desc = could not find container \"5d2b4999d29de02d8d493c74e5d5833c4405d5cb78963897853aa4a94bbdac9c\": container with ID starting with 5d2b4999d29de02d8d493c74e5d5833c4405d5cb78963897853aa4a94bbdac9c not found: ID does not exist"
Oct 30 22:16:40.794596 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.794518    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"]
Oct 30 22:16:40.797626 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.797598    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-80c366/distinct-w1-80c366-kserve-756cddd86b-w6vm2"]
Oct 30 22:16:40.950925 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.950890    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"]
Oct 30 22:16:40.955148 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:40.955109    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-80c366/premium-w1-80c366-kserve-66b94476f9-hgms6"]
Oct 30 22:16:41.447162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:41.447129    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" path="/var/lib/kubelet/pods/a63b784e-dd7f-4c97-b1f2-40644ef3d5e6/volumes"
Oct 30 22:16:41.447610 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:41.447595    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" path="/var/lib/kubelet/pods/e920b103-cdeb-4a9a-ac7f-447d89d5033f/volumes"
Oct 30 22:16:42.078027 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:42.077989    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:46.251586 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.251551    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"]
Oct 30 22:16:46.252264 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252218    5586 cpu_manager.go:414] "containerMap: removing container" podUID="7db6ba06-6961-482a-8453-8ee12cbbb85c" containerName="istio-proxy"
Oct 30 22:16:46.252264 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252259    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7db6ba06-6961-482a-8453-8ee12cbbb85c" containerName="istio-proxy"
Oct 30 22:16:46.252470 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252313    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main"
Oct 30 22:16:46.252470 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252321    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main"
Oct 30 22:16:46.252470 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252333    5586 cpu_manager.go:414] "containerMap: removing container" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main"
Oct 30 22:16:46.252470 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252340    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main"
Oct 30 22:16:46.252470 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252360    5586 cpu_manager.go:414] "containerMap: removing container" podUID="07444e4a-f484-4ff4-bffd-9c3f329fca00" containerName="cleanup"
Oct 30 22:16:46.252470 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252367    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07444e4a-f484-4ff4-bffd-9c3f329fca00" containerName="cleanup"
Oct 30 22:16:46.252470 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252389    5586 cpu_manager.go:414] "containerMap: removing container" podUID="6e70ec99-2141-4616-b598-84675558b246" containerName="cleanup"
Oct 30 22:16:46.252470 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252395    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e70ec99-2141-4616-b598-84675558b246" containerName="cleanup"
Oct 30 22:16:46.252868 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252529    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="e920b103-cdeb-4a9a-ac7f-447d89d5033f" containerName="main"
Oct 30 22:16:46.252868 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252550    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a89955b-47d7-4a3a-895f-6c3f69b695aa" containerName="istio-proxy"
Oct 30 22:16:46.252868 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252560    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e70ec99-2141-4616-b598-84675558b246" containerName="cleanup"
Oct 30 22:16:46.252868 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252570    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="7db6ba06-6961-482a-8453-8ee12cbbb85c" containerName="istio-proxy"
Oct 30 22:16:46.252868 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252586    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="003da65f-5b3a-4735-bf4d-8f064ad48d96" containerName="cleanup"
Oct 30 22:16:46.252868 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.252594    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="a63b784e-dd7f-4c97-b1f2-40644ef3d5e6" containerName="main"
Oct 30 22:16:46.256055 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.256033    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"
Oct 30 22:16:46.256994 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.256967    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g" podUID="ede829db-672f-49fa-9b86-00d130c21560" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:16:46.257199 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.257176    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48" podUID="1fe6d48f-81e8-4900-8249-0a73a66fc86b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:16:46.314711 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.314672    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"]
Oct 30 22:16:46.360092 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.360060    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:16:46.360280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.360153    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:16:46.360601 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.360579    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f48bn\" (UniqueName: \"kubernetes.io/projected/7ce0de86-fc7f-4cb5-8181-c20e8917f939-kube-api-access-f48bn\") pod \"maas-api-revoke-sub-912d72be-z75rd\" (UID: \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"
Oct 30 22:16:46.360719 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.360705    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ce0de86-fc7f-4cb5-8181-c20e8917f939-maas-api-xks-ca\") pod \"maas-api-revoke-sub-912d72be-z75rd\" (UID: \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"
Oct 30 22:16:46.360768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.360756    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ce0de86-fc7f-4cb5-8181-c20e8917f939-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-912d72be-z75rd\" (UID: \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"
Oct 30 22:16:46.461531 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.461495    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ce0de86-fc7f-4cb5-8181-c20e8917f939-maas-api-xks-ca\") pod \"maas-api-revoke-sub-912d72be-z75rd\" (UID: \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"
Oct 30 22:16:46.461726 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.461536    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ce0de86-fc7f-4cb5-8181-c20e8917f939-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-912d72be-z75rd\" (UID: \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"
Oct 30 22:16:46.461726 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.461581    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f48bn\" (UniqueName: \"kubernetes.io/projected/7ce0de86-fc7f-4cb5-8181-c20e8917f939-kube-api-access-f48bn\") pod \"maas-api-revoke-sub-912d72be-z75rd\" (UID: \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"
Oct 30 22:16:46.462193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.462161    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ce0de86-fc7f-4cb5-8181-c20e8917f939-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-912d72be-z75rd\" (UID: \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"
Oct 30 22:16:46.463980 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.463958    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ce0de86-fc7f-4cb5-8181-c20e8917f939-maas-api-xks-ca\") pod \"maas-api-revoke-sub-912d72be-z75rd\" (UID: \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"
Oct 30 22:16:46.518161 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.518099    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f48bn\" (UniqueName: \"kubernetes.io/projected/7ce0de86-fc7f-4cb5-8181-c20e8917f939-kube-api-access-f48bn\") pod \"maas-api-revoke-sub-912d72be-z75rd\" (UID: \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"
Oct 30 22:16:46.570029 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.570003    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"
Oct 30 22:16:46.947334 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:46.947308    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"]
Oct 30 22:16:46.949095 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:16:46.949069    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce0de86_fc7f_4cb5_8181_c20e8917f939.slice/crio-12773315c1dc26e0f824f6b845cee146e452c742fb2809d58d25fd989a7dcf4f WatchSource:0}: Error finding container 12773315c1dc26e0f824f6b845cee146e452c742fb2809d58d25fd989a7dcf4f: Status 404 returned error can't find the container with id 12773315c1dc26e0f824f6b845cee146e452c742fb2809d58d25fd989a7dcf4f
Oct 30 22:16:47.087038 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:47.086998    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:47.680504 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:47.680464    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd" event={"ID":"7ce0de86-fc7f-4cb5-8181-c20e8917f939","Type":"ContainerStarted","Data":"e03a5b9bc52919e5d140ab8e598d2e06eb5820d542c188031d3ebb275463f7d6"}
Oct 30 22:16:47.681003 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:47.680533    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd" event={"ID":"7ce0de86-fc7f-4cb5-8181-c20e8917f939","Type":"ContainerStarted","Data":"12773315c1dc26e0f824f6b845cee146e452c742fb2809d58d25fd989a7dcf4f"}
Oct 30 22:16:47.808708 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:47.808651    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd" podStartSLOduration=1.8086353069999999 podStartE2EDuration="1.808635307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:16:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:16:47.805228974 +0000 UTC m=+2550.477132909" watchObservedRunningTime="2026-10-30 22:16:47.808635307 +0000 UTC m=+2550.480539294"
Oct 30 22:16:48.362103 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:16:48.362058    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:16:48.688409 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:48.688372    5586 generic.go:356] "Generic (PLEG): container finished" podID="7ce0de86-fc7f-4cb5-8181-c20e8917f939" containerID="e03a5b9bc52919e5d140ab8e598d2e06eb5820d542c188031d3ebb275463f7d6" exitCode=0
Oct 30 22:16:48.688830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:48.688453    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd" event={"ID":"7ce0de86-fc7f-4cb5-8181-c20e8917f939","Type":"ContainerDied","Data":"e03a5b9bc52919e5d140ab8e598d2e06eb5820d542c188031d3ebb275463f7d6"}
Oct 30 22:16:49.841814 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:49.841784    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"
Oct 30 22:16:50.002227 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:50.002133    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ce0de86-fc7f-4cb5-8181-c20e8917f939-kube-api-access-f48bn\" (UniqueName: \"kubernetes.io/projected/7ce0de86-fc7f-4cb5-8181-c20e8917f939-kube-api-access-f48bn\") pod \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\" (UID: \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\") "
Oct 30 22:16:50.002382 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:50.002263    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ce0de86-fc7f-4cb5-8181-c20e8917f939-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ce0de86-fc7f-4cb5-8181-c20e8917f939-maas-api-xks-ca\") pod \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\" (UID: \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\") "
Oct 30 22:16:50.002382 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:50.002305    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ce0de86-fc7f-4cb5-8181-c20e8917f939-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ce0de86-fc7f-4cb5-8181-c20e8917f939-maas-api-openshift-ca\") pod \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\" (UID: \"7ce0de86-fc7f-4cb5-8181-c20e8917f939\") "
Oct 30 22:16:50.002681 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:50.002646    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce0de86-fc7f-4cb5-8181-c20e8917f939-maas-api-openshift-ca" pod "7ce0de86-fc7f-4cb5-8181-c20e8917f939" (UID: "7ce0de86-fc7f-4cb5-8181-c20e8917f939"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:50.002850 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:50.002835    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ce0de86-fc7f-4cb5-8181-c20e8917f939-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:50.004948 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:50.004920    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ce0de86-fc7f-4cb5-8181-c20e8917f939-maas-api-xks-ca" pod "7ce0de86-fc7f-4cb5-8181-c20e8917f939" (UID: "7ce0de86-fc7f-4cb5-8181-c20e8917f939"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:16:50.004948 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:50.004941    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce0de86-fc7f-4cb5-8181-c20e8917f939-kube-api-access-f48bn" pod "7ce0de86-fc7f-4cb5-8181-c20e8917f939" (UID: "7ce0de86-fc7f-4cb5-8181-c20e8917f939"). InnerVolumeSpecName "kube-api-access-f48bn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:50.104422 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:50.104383    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f48bn\" (UniqueName: \"kubernetes.io/projected/7ce0de86-fc7f-4cb5-8181-c20e8917f939-kube-api-access-f48bn\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:50.104422 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:50.104416    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ce0de86-fc7f-4cb5-8181-c20e8917f939-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:50.702605 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:50.702571    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"
Oct 30 22:16:50.702807 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:50.702607    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd" event={"ID":"7ce0de86-fc7f-4cb5-8181-c20e8917f939","Type":"ContainerDied","Data":"12773315c1dc26e0f824f6b845cee146e452c742fb2809d58d25fd989a7dcf4f"}
Oct 30 22:16:50.702807 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:50.702658    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12773315c1dc26e0f824f6b845cee146e452c742fb2809d58d25fd989a7dcf4f"
Oct 30 22:16:51.878543 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:51.878505    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:55.735054 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:55.735014    5586 generic.go:356] "Generic (PLEG): container finished" podID="ede829db-672f-49fa-9b86-00d130c21560" containerID="98dd6319631f816a0f877ad3a18a563734979b0ab0dadee09dd0b36ba8d08fd0" exitCode=0
Oct 30 22:16:55.735479 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:55.735084    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g" event={"ID":"ede829db-672f-49fa-9b86-00d130c21560","Type":"ContainerDied","Data":"98dd6319631f816a0f877ad3a18a563734979b0ab0dadee09dd0b36ba8d08fd0"}
Oct 30 22:16:55.829244 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:55.829156    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"
Oct 30 22:16:55.866613 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:55.866581    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ede829db-672f-49fa-9b86-00d130c21560-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ede829db-672f-49fa-9b86-00d130c21560-plugins-config-volume\") pod \"ede829db-672f-49fa-9b86-00d130c21560\" (UID: \"ede829db-672f-49fa-9b86-00d130c21560\") "
Oct 30 22:16:55.866822 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:55.866759    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ede829db-672f-49fa-9b86-00d130c21560-kube-api-access-rrkrw\" (UniqueName: \"kubernetes.io/projected/ede829db-672f-49fa-9b86-00d130c21560-kube-api-access-rrkrw\") pod \"ede829db-672f-49fa-9b86-00d130c21560\" (UID: \"ede829db-672f-49fa-9b86-00d130c21560\") "
Oct 30 22:16:55.867026 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:55.866994    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ede829db-672f-49fa-9b86-00d130c21560-plugins-config-volume" pod "ede829db-672f-49fa-9b86-00d130c21560" (UID: "ede829db-672f-49fa-9b86-00d130c21560"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:55.867227 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:55.867205    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ede829db-672f-49fa-9b86-00d130c21560-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:55.869223 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:55.869187    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ede829db-672f-49fa-9b86-00d130c21560-kube-api-access-rrkrw" pod "ede829db-672f-49fa-9b86-00d130c21560" (UID: "ede829db-672f-49fa-9b86-00d130c21560"). InnerVolumeSpecName "kube-api-access-rrkrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:55.905695 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:55.905671    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"
Oct 30 22:16:55.968070 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:55.968018    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fe6d48f-81e8-4900-8249-0a73a66fc86b-kube-api-access-zhgfs\" (UniqueName: \"kubernetes.io/projected/1fe6d48f-81e8-4900-8249-0a73a66fc86b-kube-api-access-zhgfs\") pod \"1fe6d48f-81e8-4900-8249-0a73a66fc86b\" (UID: \"1fe6d48f-81e8-4900-8249-0a73a66fc86b\") "
Oct 30 22:16:55.968304 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:55.968181    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1fe6d48f-81e8-4900-8249-0a73a66fc86b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fe6d48f-81e8-4900-8249-0a73a66fc86b-plugins-config-volume\") pod \"1fe6d48f-81e8-4900-8249-0a73a66fc86b\" (UID: \"1fe6d48f-81e8-4900-8249-0a73a66fc86b\") "
Oct 30 22:16:55.968629 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:55.968599    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fe6d48f-81e8-4900-8249-0a73a66fc86b-plugins-config-volume" pod "1fe6d48f-81e8-4900-8249-0a73a66fc86b" (UID: "1fe6d48f-81e8-4900-8249-0a73a66fc86b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:55.968712 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:55.968616    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrkrw\" (UniqueName: \"kubernetes.io/projected/ede829db-672f-49fa-9b86-00d130c21560-kube-api-access-rrkrw\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:55.970115 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:55.970087    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fe6d48f-81e8-4900-8249-0a73a66fc86b-kube-api-access-zhgfs" pod "1fe6d48f-81e8-4900-8249-0a73a66fc86b" (UID: "1fe6d48f-81e8-4900-8249-0a73a66fc86b"). InnerVolumeSpecName "kube-api-access-zhgfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:56.069493 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.069461    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhgfs\" (UniqueName: \"kubernetes.io/projected/1fe6d48f-81e8-4900-8249-0a73a66fc86b-kube-api-access-zhgfs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:56.069493 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.069498    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fe6d48f-81e8-4900-8249-0a73a66fc86b-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:56.689861 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.689808    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:16:56.746396 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.746366    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"
Oct 30 22:16:56.746396 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.746382    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g" event={"ID":"ede829db-672f-49fa-9b86-00d130c21560","Type":"ContainerDied","Data":"3b405133446a7b8bd6c479be252950c8271beea2bd1ae74597d9f15ed10b8d48"}
Oct 30 22:16:56.746972 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.746449    5586 scope.go:122] "RemoveContainer" containerID="98dd6319631f816a0f877ad3a18a563734979b0ab0dadee09dd0b36ba8d08fd0"
Oct 30 22:16:56.748406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.748227    5586 generic.go:356] "Generic (PLEG): container finished" podID="1fe6d48f-81e8-4900-8249-0a73a66fc86b" containerID="efc2e975250cfaedd0ce1f11668076a2d16f9e1a29a18b20cf8fee9169242e13" exitCode=0
Oct 30 22:16:56.748406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.748297    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48" event={"ID":"1fe6d48f-81e8-4900-8249-0a73a66fc86b","Type":"ContainerDied","Data":"efc2e975250cfaedd0ce1f11668076a2d16f9e1a29a18b20cf8fee9169242e13"}
Oct 30 22:16:56.748406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.748360    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48" event={"ID":"1fe6d48f-81e8-4900-8249-0a73a66fc86b","Type":"ContainerDied","Data":"17f77054f542f04d966240ea133007a72ab11e9c06f538949a15d221c7a15f2a"}
Oct 30 22:16:56.748406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.748366    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"
Oct 30 22:16:56.763642 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.763616    5586 scope.go:122] "RemoveContainer" containerID="efc2e975250cfaedd0ce1f11668076a2d16f9e1a29a18b20cf8fee9169242e13"
Oct 30 22:16:56.775183 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.775157    5586 scope.go:122] "RemoveContainer" containerID="efc2e975250cfaedd0ce1f11668076a2d16f9e1a29a18b20cf8fee9169242e13"
Oct 30 22:16:56.775608 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:16:56.775585    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efc2e975250cfaedd0ce1f11668076a2d16f9e1a29a18b20cf8fee9169242e13\": container with ID starting with efc2e975250cfaedd0ce1f11668076a2d16f9e1a29a18b20cf8fee9169242e13 not found: ID does not exist" containerID="efc2e975250cfaedd0ce1f11668076a2d16f9e1a29a18b20cf8fee9169242e13"
Oct 30 22:16:56.775681 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.775618    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efc2e975250cfaedd0ce1f11668076a2d16f9e1a29a18b20cf8fee9169242e13"} err="failed to get container status \"efc2e975250cfaedd0ce1f11668076a2d16f9e1a29a18b20cf8fee9169242e13\": rpc error: code = NotFound desc = could not find container \"efc2e975250cfaedd0ce1f11668076a2d16f9e1a29a18b20cf8fee9169242e13\": container with ID starting with efc2e975250cfaedd0ce1f11668076a2d16f9e1a29a18b20cf8fee9169242e13 not found: ID does not exist"
Oct 30 22:16:56.911047 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.911014    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"]
Oct 30 22:16:56.913485 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:56.913463    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-80c366-78579d4fff-kzl48"]
Oct 30 22:16:57.033516 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:57.033472    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"]
Oct 30 22:16:57.044849 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:57.044813    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-80c366-58948dbb9c-g7x6g"]
Oct 30 22:16:57.450699 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:57.450655    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1fe6d48f-81e8-4900-8249-0a73a66fc86b" path="/var/lib/kubelet/pods/1fe6d48f-81e8-4900-8249-0a73a66fc86b/volumes"
Oct 30 22:16:57.451178 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:16:57.451158    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ede829db-672f-49fa-9b86-00d130c21560" path="/var/lib/kubelet/pods/ede829db-672f-49fa-9b86-00d130c21560/volumes"
Oct 30 22:17:00.888523 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:00.888483    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:01.048185 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:01.048142    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:01.613522 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:01.613486    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:03.208553 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:03.208514    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:06.394400 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:06.394363    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:06.946292 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:06.946225    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:07.102662 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:07.102627    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:09.336548 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:09.336513    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:09.497448 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:09.497408    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:11.202368 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:11.202303    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:15.983425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:15.983377    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:18.361216 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:17:18.361172    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:17:20.203917 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:20.203872    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:21.771652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:21.771598    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:21.980053 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:21.980016    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:22.145172 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:22.145079    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:22.706263 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:22.706209    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:27.121343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:27.121300    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:30.190497 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:30.190455    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:30.399637 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:30.399572    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:30.576242 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:30.576144    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"]
Oct 30 22:17:30.576403 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:30.576358    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk" podUID="e00cca75-dfc2-4231-ad1c-61c7069faa38" containerName="istio-proxy" containerID="cri-o://965e41df6b94f7294ea27d1d9f3c3c924b5bae9d16993cab243fe45757213f74" gracePeriod=30
Oct 30 22:17:31.471912 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:31.471863    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:32.292289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:32.292213    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:35.840225 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.840199    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 22:17:35.933955 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.933924    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-envoy\") pod \"e00cca75-dfc2-4231-ad1c-61c7069faa38\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") "
Oct 30 22:17:35.934157 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.934049    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-podinfo\") pod \"e00cca75-dfc2-4231-ad1c-61c7069faa38\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") "
Oct 30 22:17:35.934157 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.934087    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e00cca75-dfc2-4231-ad1c-61c7069faa38-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00cca75-dfc2-4231-ad1c-61c7069faa38-istiod-ca-cert\") pod \"e00cca75-dfc2-4231-ad1c-61c7069faa38\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") "
Oct 30 22:17:35.934157 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.934135    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-token\" (UniqueName: \"kubernetes.io/projected/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-token\") pod \"e00cca75-dfc2-4231-ad1c-61c7069faa38\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") "
Oct 30 22:17:35.934580 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.934168    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-workload-socket\") pod \"e00cca75-dfc2-4231-ad1c-61c7069faa38\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") "
Oct 30 22:17:35.934580 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.934199    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-workload-certs\") pod \"e00cca75-dfc2-4231-ad1c-61c7069faa38\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") "
Oct 30 22:17:35.934580 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.934227    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-data\") pod \"e00cca75-dfc2-4231-ad1c-61c7069faa38\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") "
Oct 30 22:17:35.934580 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.934305    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e00cca75-dfc2-4231-ad1c-61c7069faa38-kube-api-access-xsssd\" (UniqueName: \"kubernetes.io/projected/e00cca75-dfc2-4231-ad1c-61c7069faa38-kube-api-access-xsssd\") pod \"e00cca75-dfc2-4231-ad1c-61c7069faa38\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") "
Oct 30 22:17:35.934580 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.934355    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-credential-socket\") pod \"e00cca75-dfc2-4231-ad1c-61c7069faa38\" (UID: \"e00cca75-dfc2-4231-ad1c-61c7069faa38\") "
Oct 30 22:17:35.934857 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.934606    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e00cca75-dfc2-4231-ad1c-61c7069faa38-istiod-ca-cert" pod "e00cca75-dfc2-4231-ad1c-61c7069faa38" (UID: "e00cca75-dfc2-4231-ad1c-61c7069faa38"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:35.934857 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.934603    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-data" pod "e00cca75-dfc2-4231-ad1c-61c7069faa38" (UID: "e00cca75-dfc2-4231-ad1c-61c7069faa38"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:35.934857 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.934745    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-workload-socket" pod "e00cca75-dfc2-4231-ad1c-61c7069faa38" (UID: "e00cca75-dfc2-4231-ad1c-61c7069faa38"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:35.935036 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.934887    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-workload-certs" pod "e00cca75-dfc2-4231-ad1c-61c7069faa38" (UID: "e00cca75-dfc2-4231-ad1c-61c7069faa38"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:35.935036 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.934903    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-credential-socket" pod "e00cca75-dfc2-4231-ad1c-61c7069faa38" (UID: "e00cca75-dfc2-4231-ad1c-61c7069faa38"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:35.935036 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.935007    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00cca75-dfc2-4231-ad1c-61c7069faa38-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:35.935036 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.935027    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:35.935036 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.935036    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:35.936467 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.936437    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-podinfo" pod "e00cca75-dfc2-4231-ad1c-61c7069faa38" (UID: "e00cca75-dfc2-4231-ad1c-61c7069faa38"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:17:35.936467 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.936449    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-envoy" pod "e00cca75-dfc2-4231-ad1c-61c7069faa38" (UID: "e00cca75-dfc2-4231-ad1c-61c7069faa38"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:35.936655 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.936482    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-token" pod "e00cca75-dfc2-4231-ad1c-61c7069faa38" (UID: "e00cca75-dfc2-4231-ad1c-61c7069faa38"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:35.936719 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:35.936670    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e00cca75-dfc2-4231-ad1c-61c7069faa38-kube-api-access-xsssd" pod "e00cca75-dfc2-4231-ad1c-61c7069faa38" (UID: "e00cca75-dfc2-4231-ad1c-61c7069faa38"). InnerVolumeSpecName "kube-api-access-xsssd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:36.003045 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.003007    5586 generic.go:356] "Generic (PLEG): container finished" podID="e00cca75-dfc2-4231-ad1c-61c7069faa38" containerID="965e41df6b94f7294ea27d1d9f3c3c924b5bae9d16993cab243fe45757213f74" exitCode=0
Oct 30 22:17:36.003208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.003062    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"
Oct 30 22:17:36.003208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.003088    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk" event={"ID":"e00cca75-dfc2-4231-ad1c-61c7069faa38","Type":"ContainerDied","Data":"965e41df6b94f7294ea27d1d9f3c3c924b5bae9d16993cab243fe45757213f74"}
Oct 30 22:17:36.003208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.003144    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk" event={"ID":"e00cca75-dfc2-4231-ad1c-61c7069faa38","Type":"ContainerDied","Data":"2b12a3f966b483859004f4baa78482ca55f2a57bfb365c8a00ae51eeb4ef376b"}
Oct 30 22:17:36.003208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.003165    5586 scope.go:122] "RemoveContainer" containerID="965e41df6b94f7294ea27d1d9f3c3c924b5bae9d16993cab243fe45757213f74"
Oct 30 22:17:36.020759 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.020736    5586 scope.go:122] "RemoveContainer" containerID="965e41df6b94f7294ea27d1d9f3c3c924b5bae9d16993cab243fe45757213f74"
Oct 30 22:17:36.021130 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:17:36.021108    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"965e41df6b94f7294ea27d1d9f3c3c924b5bae9d16993cab243fe45757213f74\": container with ID starting with 965e41df6b94f7294ea27d1d9f3c3c924b5bae9d16993cab243fe45757213f74 not found: ID does not exist" containerID="965e41df6b94f7294ea27d1d9f3c3c924b5bae9d16993cab243fe45757213f74"
Oct 30 22:17:36.021219 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.021143    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"965e41df6b94f7294ea27d1d9f3c3c924b5bae9d16993cab243fe45757213f74"} err="failed to get container status \"965e41df6b94f7294ea27d1d9f3c3c924b5bae9d16993cab243fe45757213f74\": rpc error: code = NotFound desc = could not find container \"965e41df6b94f7294ea27d1d9f3c3c924b5bae9d16993cab243fe45757213f74\": container with ID starting with 965e41df6b94f7294ea27d1d9f3c3c924b5bae9d16993cab243fe45757213f74 not found: ID does not exist"
Oct 30 22:17:36.036915 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.036885    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:36.037022 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.036920    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsssd\" (UniqueName: \"kubernetes.io/projected/e00cca75-dfc2-4231-ad1c-61c7069faa38-kube-api-access-xsssd\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:36.037022 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.036937    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:36.037022 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.036953    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:36.037022 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.036967    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:36.037022 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.036980    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e00cca75-dfc2-4231-ad1c-61c7069faa38-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:36.176999 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.176935    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"]
Oct 30 22:17:36.180255 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:36.180216    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-80c366-openshift-default-666b7db455-8tpgk"]
Oct 30 22:17:37.169554 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:37.169508    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:37.446958 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:37.446882    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e00cca75-dfc2-4231-ad1c-61c7069faa38" path="/var/lib/kubelet/pods/e00cca75-dfc2-4231-ad1c-61c7069faa38/volumes"
Oct 30 22:17:41.869201 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:41.869152    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:46.008152 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:46.008108    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:46.158660 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:46.158612    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:46.589448 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:46.589408    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:48.361381 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:17:48.361338    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:17:51.533938 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:51.526513    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:53.211276 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:53.211223    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:53.436118 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:53.436076    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:17:56.388019 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:17:56.387981    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:01.301657 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:01.301432    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:06.183318 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:06.183282    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:07.697559 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:07.697517    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:11.071396 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:11.071359    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:15.987937 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:15.987894    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:18.361939 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:18:18.361896    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:18:20.910013 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:20.909971    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:22.267067 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.267023    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"]
Oct 30 22:18:22.267629 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.267343    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd" podUID="1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc" containerName="discovery" containerID="cri-o://eac86fb5028cf63f0da6909e325dbdb5cddfbf48d532a66d299a08472aebca62" gracePeriod=30
Oct 30 22:18:22.538741 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.538707    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 22:18:22.668329 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.668288    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-token\" (UniqueName: \"kubernetes.io/projected/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-token\") pod \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") "
Oct 30 22:18:22.668520 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.668352    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-kube-api-access-4pr4f\" (UniqueName: \"kubernetes.io/projected/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-kube-api-access-4pr4f\") pod \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") "
Oct 30 22:18:22.668520 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.668387    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-csr-ca-configmap\") pod \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") "
Oct 30 22:18:22.668520 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.668415    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-kubeconfig\") pod \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") "
Oct 30 22:18:22.668520 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.668458    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-csr-dns-cert\") pod \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") "
Oct 30 22:18:22.668741 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.668538    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-cacerts\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-cacerts\") pod \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") "
Oct 30 22:18:22.668741 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.668564    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-local-certs\") pod \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\" (UID: \"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc\") "
Oct 30 22:18:22.669156 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.669109    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-csr-ca-configmap" pod "1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc" (UID: "1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:22.671274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.671216    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-token" pod "1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc" (UID: "1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:22.671748 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.671726    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-kube-api-access-4pr4f" pod "1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc" (UID: "1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc"). InnerVolumeSpecName "kube-api-access-4pr4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:22.672115 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.672087    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-kubeconfig" pod "1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc" (UID: "1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:22.672189 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.672105    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-cacerts" pod "1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc" (UID: "1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:22.672189 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.672158    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-csr-dns-cert" pod "1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc" (UID: "1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:22.676566 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.676538    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-local-certs" pod "1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc" (UID: "1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:22.769774 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.769736    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.769774 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.769770    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pr4f\" (UniqueName: \"kubernetes.io/projected/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-kube-api-access-4pr4f\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.769774 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.769781    5586 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-csr-ca-configmap\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.770070 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.769791    5586 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-kubeconfig\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.770070 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.769800    5586 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-istio-csr-dns-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.770070 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.769810    5586 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-cacerts\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.770070 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:22.769819    5586 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc-local-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:23.301609 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:23.301567    5586 generic.go:356] "Generic (PLEG): container finished" podID="1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc" containerID="eac86fb5028cf63f0da6909e325dbdb5cddfbf48d532a66d299a08472aebca62" exitCode=0
Oct 30 22:18:23.302119 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:23.301620    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"
Oct 30 22:18:23.302119 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:23.301657    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd" event={"ID":"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc","Type":"ContainerDied","Data":"eac86fb5028cf63f0da6909e325dbdb5cddfbf48d532a66d299a08472aebca62"}
Oct 30 22:18:23.302119 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:23.301729    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd" event={"ID":"1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc","Type":"ContainerDied","Data":"8e383fc67d673554a2df1b3178fd3694a61614d30509d2ab4326fdb5e57b6efe"}
Oct 30 22:18:23.302119 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:23.301761    5586 scope.go:122] "RemoveContainer" containerID="eac86fb5028cf63f0da6909e325dbdb5cddfbf48d532a66d299a08472aebca62"
Oct 30 22:18:23.312926 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:23.312905    5586 scope.go:122] "RemoveContainer" containerID="eac86fb5028cf63f0da6909e325dbdb5cddfbf48d532a66d299a08472aebca62"
Oct 30 22:18:23.313221 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:18:23.313205    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eac86fb5028cf63f0da6909e325dbdb5cddfbf48d532a66d299a08472aebca62\": container with ID starting with eac86fb5028cf63f0da6909e325dbdb5cddfbf48d532a66d299a08472aebca62 not found: ID does not exist" containerID="eac86fb5028cf63f0da6909e325dbdb5cddfbf48d532a66d299a08472aebca62"
Oct 30 22:18:23.313283 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:23.313242    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eac86fb5028cf63f0da6909e325dbdb5cddfbf48d532a66d299a08472aebca62"} err="failed to get container status \"eac86fb5028cf63f0da6909e325dbdb5cddfbf48d532a66d299a08472aebca62\": rpc error: code = NotFound desc = could not find container \"eac86fb5028cf63f0da6909e325dbdb5cddfbf48d532a66d299a08472aebca62\": container with ID starting with eac86fb5028cf63f0da6909e325dbdb5cddfbf48d532a66d299a08472aebca62 not found: ID does not exist"
Oct 30 22:18:23.466488 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:23.466456    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"]
Oct 30 22:18:23.515743 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:23.515705    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-4z8hd"]
Oct 30 22:18:25.447388 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:25.447356    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc" path="/var/lib/kubelet/pods/1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc/volumes"
Oct 30 22:18:25.799349 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:25.799255    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:30.750046 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:30.750005    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:30.928592 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:30.928553    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:31.109086 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.108988    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:31.271544 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.271504    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"]
Oct 30 22:18:31.272019 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272006    5586 cpu_manager.go:414] "containerMap: removing container" podUID="1fe6d48f-81e8-4900-8249-0a73a66fc86b" containerName="payload-pre-processing"
Oct 30 22:18:31.272075 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272020    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fe6d48f-81e8-4900-8249-0a73a66fc86b" containerName="payload-pre-processing"
Oct 30 22:18:31.272075 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272046    5586 cpu_manager.go:414] "containerMap: removing container" podUID="e00cca75-dfc2-4231-ad1c-61c7069faa38" containerName="istio-proxy"
Oct 30 22:18:31.272075 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272053    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e00cca75-dfc2-4231-ad1c-61c7069faa38" containerName="istio-proxy"
Oct 30 22:18:31.272075 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272066    5586 cpu_manager.go:414] "containerMap: removing container" podUID="7ce0de86-fc7f-4cb5-8181-c20e8917f939" containerName="revoke-keys"
Oct 30 22:18:31.272075 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272071    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ce0de86-fc7f-4cb5-8181-c20e8917f939" containerName="revoke-keys"
Oct 30 22:18:31.272258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272081    5586 cpu_manager.go:414] "containerMap: removing container" podUID="1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc" containerName="discovery"
Oct 30 22:18:31.272258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272086    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc" containerName="discovery"
Oct 30 22:18:31.272258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272096    5586 cpu_manager.go:414] "containerMap: removing container" podUID="ede829db-672f-49fa-9b86-00d130c21560" containerName="payload-processing"
Oct 30 22:18:31.272258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272101    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ede829db-672f-49fa-9b86-00d130c21560" containerName="payload-processing"
Oct 30 22:18:31.272258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272163    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="1fe6d48f-81e8-4900-8249-0a73a66fc86b" containerName="payload-pre-processing"
Oct 30 22:18:31.272258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272176    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="e00cca75-dfc2-4231-ad1c-61c7069faa38" containerName="istio-proxy"
Oct 30 22:18:31.272258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272184    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ce0de86-fc7f-4cb5-8181-c20e8917f939" containerName="revoke-keys"
Oct 30 22:18:31.272258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272189    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b3d579e-7b2e-46bb-87a5-0bc19e8d00fc" containerName="discovery"
Oct 30 22:18:31.272258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.272195    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="ede829db-672f-49fa-9b86-00d130c21560" containerName="payload-processing"
Oct 30 22:18:31.277258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.277223    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"
Oct 30 22:18:31.335063 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.335035    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:18:31.335290 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.335272    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:18:31.337178 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.337152    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"]
Oct 30 22:18:31.364991 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.364904    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d37c6852-c3a7-4827-beb7-62b415bd6308-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj\" (UID: \"d37c6852-c3a7-4827-beb7-62b415bd6308\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"
Oct 30 22:18:31.365153 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.365100    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkp8c\" (UniqueName: \"kubernetes.io/projected/d37c6852-c3a7-4827-beb7-62b415bd6308-kube-api-access-zkp8c\") pod \"maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj\" (UID: \"d37c6852-c3a7-4827-beb7-62b415bd6308\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"
Oct 30 22:18:31.365153 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.365147    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d37c6852-c3a7-4827-beb7-62b415bd6308-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj\" (UID: \"d37c6852-c3a7-4827-beb7-62b415bd6308\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"
Oct 30 22:18:31.466137 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.466100    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d37c6852-c3a7-4827-beb7-62b415bd6308-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj\" (UID: \"d37c6852-c3a7-4827-beb7-62b415bd6308\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"
Oct 30 22:18:31.466388 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.466194    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkp8c\" (UniqueName: \"kubernetes.io/projected/d37c6852-c3a7-4827-beb7-62b415bd6308-kube-api-access-zkp8c\") pod \"maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj\" (UID: \"d37c6852-c3a7-4827-beb7-62b415bd6308\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"
Oct 30 22:18:31.466388 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.466222    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d37c6852-c3a7-4827-beb7-62b415bd6308-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj\" (UID: \"d37c6852-c3a7-4827-beb7-62b415bd6308\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"
Oct 30 22:18:31.466889 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.466861    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d37c6852-c3a7-4827-beb7-62b415bd6308-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj\" (UID: \"d37c6852-c3a7-4827-beb7-62b415bd6308\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"
Oct 30 22:18:31.468651 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.468632    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d37c6852-c3a7-4827-beb7-62b415bd6308-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj\" (UID: \"d37c6852-c3a7-4827-beb7-62b415bd6308\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"
Oct 30 22:18:31.523966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.523935    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkp8c\" (UniqueName: \"kubernetes.io/projected/d37c6852-c3a7-4827-beb7-62b415bd6308-kube-api-access-zkp8c\") pod \"maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj\" (UID: \"d37c6852-c3a7-4827-beb7-62b415bd6308\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"
Oct 30 22:18:31.590898 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.590860    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"
Oct 30 22:18:31.971985 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:31.971948    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"]
Oct 30 22:18:31.973814 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:18:31.973783    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd37c6852_c3a7_4827_beb7_62b415bd6308.slice/crio-758421676bea0da9741d986d7c7d0bb20804a1ef534424468961fafd2fcc5bcc WatchSource:0}: Error finding container 758421676bea0da9741d986d7c7d0bb20804a1ef534424468961fafd2fcc5bcc: Status 404 returned error can't find the container with id 758421676bea0da9741d986d7c7d0bb20804a1ef534424468961fafd2fcc5bcc
Oct 30 22:18:32.359545 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:32.359500    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj" event={"ID":"d37c6852-c3a7-4827-beb7-62b415bd6308","Type":"ContainerStarted","Data":"283191a7da3443481d59b9bb66f0f323df2c46829755beb219d69505d89e3fd9"}
Oct 30 22:18:32.359739 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:32.359570    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj" event={"ID":"d37c6852-c3a7-4827-beb7-62b415bd6308","Type":"ContainerStarted","Data":"758421676bea0da9741d986d7c7d0bb20804a1ef534424468961fafd2fcc5bcc"}
Oct 30 22:18:32.479951 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:32.479883    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj" podStartSLOduration=1.479867668 podStartE2EDuration="1.479867668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:32.477572294 +0000 UTC m=+2655.149476228" watchObservedRunningTime="2026-10-30 22:18:32.479867668 +0000 UTC m=+2655.151771602"
Oct 30 22:18:33.367940 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:33.367841    5586 generic.go:356] "Generic (PLEG): container finished" podID="d37c6852-c3a7-4827-beb7-62b415bd6308" containerID="283191a7da3443481d59b9bb66f0f323df2c46829755beb219d69505d89e3fd9" exitCode=0
Oct 30 22:18:33.368448 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:33.367977    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj" event={"ID":"d37c6852-c3a7-4827-beb7-62b415bd6308","Type":"ContainerDied","Data":"283191a7da3443481d59b9bb66f0f323df2c46829755beb219d69505d89e3fd9"}
Oct 30 22:18:34.510392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:34.510364    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"
Oct 30 22:18:34.599006 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:34.598967    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d37c6852-c3a7-4827-beb7-62b415bd6308-kube-api-access-zkp8c\" (UniqueName: \"kubernetes.io/projected/d37c6852-c3a7-4827-beb7-62b415bd6308-kube-api-access-zkp8c\") pod \"d37c6852-c3a7-4827-beb7-62b415bd6308\" (UID: \"d37c6852-c3a7-4827-beb7-62b415bd6308\") "
Oct 30 22:18:34.599227 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:34.599070    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d37c6852-c3a7-4827-beb7-62b415bd6308-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d37c6852-c3a7-4827-beb7-62b415bd6308-maas-api-xks-ca\") pod \"d37c6852-c3a7-4827-beb7-62b415bd6308\" (UID: \"d37c6852-c3a7-4827-beb7-62b415bd6308\") "
Oct 30 22:18:34.599227 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:34.599138    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d37c6852-c3a7-4827-beb7-62b415bd6308-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d37c6852-c3a7-4827-beb7-62b415bd6308-maas-api-openshift-ca\") pod \"d37c6852-c3a7-4827-beb7-62b415bd6308\" (UID: \"d37c6852-c3a7-4827-beb7-62b415bd6308\") "
Oct 30 22:18:34.599595 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:34.599565    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d37c6852-c3a7-4827-beb7-62b415bd6308-maas-api-openshift-ca" pod "d37c6852-c3a7-4827-beb7-62b415bd6308" (UID: "d37c6852-c3a7-4827-beb7-62b415bd6308"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:34.601152 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:34.601125    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d37c6852-c3a7-4827-beb7-62b415bd6308-kube-api-access-zkp8c" pod "d37c6852-c3a7-4827-beb7-62b415bd6308" (UID: "d37c6852-c3a7-4827-beb7-62b415bd6308"). InnerVolumeSpecName "kube-api-access-zkp8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:34.601282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:34.601181    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d37c6852-c3a7-4827-beb7-62b415bd6308-maas-api-xks-ca" pod "d37c6852-c3a7-4827-beb7-62b415bd6308" (UID: "d37c6852-c3a7-4827-beb7-62b415bd6308"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:34.700466 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:34.700421    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkp8c\" (UniqueName: \"kubernetes.io/projected/d37c6852-c3a7-4827-beb7-62b415bd6308-kube-api-access-zkp8c\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:34.700466 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:34.700457    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d37c6852-c3a7-4827-beb7-62b415bd6308-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:34.700466 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:34.700469    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d37c6852-c3a7-4827-beb7-62b415bd6308-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:35.381537 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:35.381507    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"
Oct 30 22:18:35.381758 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:35.381544    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj" event={"ID":"d37c6852-c3a7-4827-beb7-62b415bd6308","Type":"ContainerDied","Data":"758421676bea0da9741d986d7c7d0bb20804a1ef534424468961fafd2fcc5bcc"}
Oct 30 22:18:35.381758 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:35.381604    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="758421676bea0da9741d986d7c7d0bb20804a1ef534424468961fafd2fcc5bcc"
Oct 30 22:18:35.999141 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:35.999107    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:40.923734 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:40.923689    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:42.055502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:42.055467    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:42.211812 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:42.211771    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:45.800706 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:45.800672    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:48.361691 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:18:48.361657    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:18:50.591741 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:50.591696    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:55.561749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:18:55.561699    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:00.512133 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:00.512096    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:05.273954 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:05.273914    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:09.450547 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:09.450507    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:11.019496 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:11.019457    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:11.227263 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:11.227195    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:11.432111 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:11.432073    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:11.533928 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:11.533892    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"]
Oct 30 22:19:11.583245 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:11.583195    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9da607-5tfgj"]
Oct 30 22:19:12.145218 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:12.145174    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:13.460258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:13.460049    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d37c6852-c3a7-4827-beb7-62b415bd6308" path="/var/lib/kubelet/pods/d37c6852-c3a7-4827-beb7-62b415bd6308/volumes"
Oct 30 22:19:16.291628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:16.291593    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:17.086119 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:17.086077    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"]
Oct 30 22:19:17.135891 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:17.135848    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-48b5b732-7w9gs"]
Oct 30 22:19:17.138618 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:17.138584    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"]
Oct 30 22:19:17.140891 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:17.140862    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-59f6a75f-mj7mf"]
Oct 30 22:19:17.447692 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:17.447659    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b4be694-b324-4fa7-a5fd-0703c94340e5" path="/var/lib/kubelet/pods/8b4be694-b324-4fa7-a5fd-0703c94340e5/volumes"
Oct 30 22:19:17.448106 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:17.448092    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2e3476a-f9c3-42d6-bfe4-10348a474ba7" path="/var/lib/kubelet/pods/f2e3476a-f9c3-42d6-bfe4-10348a474ba7/volumes"
Oct 30 22:19:18.021584 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:18.021555    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:19:18.036786 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:18.036757    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:19:18.038785 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:18.038758    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:19:18.043745 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:18.043714    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:19:18.361141 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:19:18.361053    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:19:19.404547 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:19.404510    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:19.663479 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:19.663383    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:19.883830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:19.883794    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:21.297778 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:21.297744    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:26.234379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:26.234343    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:31.098781 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:31.098740    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:31.342604 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:31.342566    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:31.513583 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:31.513551    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:31.676772 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:31.676728    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:35.157155 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:35.157122    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:36.708469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:36.708427    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:36.974871 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:36.974778    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:37.157808 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:37.157769    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:37.674287 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:37.674221    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:42.236165 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:42.236122    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:43.954511 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:43.954472    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:44.117844 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:44.117808    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:44.280139 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:44.280055    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"]
Oct 30 22:19:44.280331 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:44.280307    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs" podUID="26432f05-beba-4093-a139-217dac85db7e" containerName="istio-proxy" containerID="cri-o://7b474842685af055af4df70c9fc33183c360aa6c54b16b78f0fad2720e923420" gracePeriod=30
Oct 30 22:19:44.302661 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:44.302625    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:45.036921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:45.036884    5586 patch_prober.go:29] interesting pod/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:19:45.037373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:45.036953    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs" podUID="26432f05-beba-4093-a139-217dac85db7e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:19:46.985170 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:46.985132    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:48.360686 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:19:48.360648    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:19:49.537524 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.537491    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:19:49.613597 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.613494    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-workload-certs\") pod \"26432f05-beba-4093-a139-217dac85db7e\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") "
Oct 30 22:19:49.613790 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.613623    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-credential-socket\") pod \"26432f05-beba-4093-a139-217dac85db7e\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") "
Oct 30 22:19:49.613790 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.613651    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-workload-socket\") pod \"26432f05-beba-4093-a139-217dac85db7e\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") "
Oct 30 22:19:49.613790 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.613710    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-istio-data\") pod \"26432f05-beba-4093-a139-217dac85db7e\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") "
Oct 30 22:19:49.613790 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.613771    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26432f05-beba-4093-a139-217dac85db7e-istio-token\" (UniqueName: \"kubernetes.io/projected/26432f05-beba-4093-a139-217dac85db7e-istio-token\") pod \"26432f05-beba-4093-a139-217dac85db7e\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") "
Oct 30 22:19:49.614019 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.613787    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-workload-certs" pod "26432f05-beba-4093-a139-217dac85db7e" (UID: "26432f05-beba-4093-a139-217dac85db7e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:49.614019 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.613812    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/26432f05-beba-4093-a139-217dac85db7e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26432f05-beba-4093-a139-217dac85db7e-istio-podinfo\") pod \"26432f05-beba-4093-a139-217dac85db7e\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") "
Oct 30 22:19:49.614019 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.613905    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/26432f05-beba-4093-a139-217dac85db7e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26432f05-beba-4093-a139-217dac85db7e-istiod-ca-cert\") pod \"26432f05-beba-4093-a139-217dac85db7e\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") "
Oct 30 22:19:49.614019 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.613948    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-credential-socket" pod "26432f05-beba-4093-a139-217dac85db7e" (UID: "26432f05-beba-4093-a139-217dac85db7e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:49.614246 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.614031    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-workload-socket" pod "26432f05-beba-4093-a139-217dac85db7e" (UID: "26432f05-beba-4093-a139-217dac85db7e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:49.614246 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.614042    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-istio-data" pod "26432f05-beba-4093-a139-217dac85db7e" (UID: "26432f05-beba-4093-a139-217dac85db7e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:49.614246 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.614055    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26432f05-beba-4093-a139-217dac85db7e-kube-api-access-nn257\" (UniqueName: \"kubernetes.io/projected/26432f05-beba-4093-a139-217dac85db7e-kube-api-access-nn257\") pod \"26432f05-beba-4093-a139-217dac85db7e\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") "
Oct 30 22:19:49.614246 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.614086    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-istio-envoy\") pod \"26432f05-beba-4093-a139-217dac85db7e\" (UID: \"26432f05-beba-4093-a139-217dac85db7e\") "
Oct 30 22:19:49.614439 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.614309    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26432f05-beba-4093-a139-217dac85db7e-istiod-ca-cert" pod "26432f05-beba-4093-a139-217dac85db7e" (UID: "26432f05-beba-4093-a139-217dac85db7e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:49.614586 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.614562    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:49.614678 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.614591    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:49.614678 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.614617    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:49.614678 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.614631    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26432f05-beba-4093-a139-217dac85db7e-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:49.614678 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.614644    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:49.616258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.616162    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-istio-envoy" pod "26432f05-beba-4093-a139-217dac85db7e" (UID: "26432f05-beba-4093-a139-217dac85db7e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:49.616406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.616273    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26432f05-beba-4093-a139-217dac85db7e-istio-token" pod "26432f05-beba-4093-a139-217dac85db7e" (UID: "26432f05-beba-4093-a139-217dac85db7e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:49.616406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.616289    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26432f05-beba-4093-a139-217dac85db7e-kube-api-access-nn257" pod "26432f05-beba-4093-a139-217dac85db7e" (UID: "26432f05-beba-4093-a139-217dac85db7e"). InnerVolumeSpecName "kube-api-access-nn257". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:49.616406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.616337    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/26432f05-beba-4093-a139-217dac85db7e-istio-podinfo" pod "26432f05-beba-4093-a139-217dac85db7e" (UID: "26432f05-beba-4093-a139-217dac85db7e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:19:49.715785 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.715744    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26432f05-beba-4093-a139-217dac85db7e-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:49.715785 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.715788    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26432f05-beba-4093-a139-217dac85db7e-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:49.715999 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.715804    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nn257\" (UniqueName: \"kubernetes.io/projected/26432f05-beba-4093-a139-217dac85db7e-kube-api-access-nn257\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:49.715999 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.715819    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26432f05-beba-4093-a139-217dac85db7e-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:49.839247 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.839195    5586 generic.go:356] "Generic (PLEG): container finished" podID="26432f05-beba-4093-a139-217dac85db7e" containerID="7b474842685af055af4df70c9fc33183c360aa6c54b16b78f0fad2720e923420" exitCode=0
Oct 30 22:19:49.839433 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.839261    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"
Oct 30 22:19:49.839433 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.839276    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs" event={"ID":"26432f05-beba-4093-a139-217dac85db7e","Type":"ContainerDied","Data":"7b474842685af055af4df70c9fc33183c360aa6c54b16b78f0fad2720e923420"}
Oct 30 22:19:49.839433 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.839334    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs" event={"ID":"26432f05-beba-4093-a139-217dac85db7e","Type":"ContainerDied","Data":"b07e36371ae7b7f8edb971b0fae871e097852555584cc68d855ef17dbad64a73"}
Oct 30 22:19:49.839433 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.839354    5586 scope.go:122] "RemoveContainer" containerID="7b474842685af055af4df70c9fc33183c360aa6c54b16b78f0fad2720e923420"
Oct 30 22:19:49.850544 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.850521    5586 scope.go:122] "RemoveContainer" containerID="82b93ee8f4e0657f07b778d752df9415be6df278addfdfb9a327430e339434cb"
Oct 30 22:19:49.861456 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.861438    5586 scope.go:122] "RemoveContainer" containerID="7b474842685af055af4df70c9fc33183c360aa6c54b16b78f0fad2720e923420"
Oct 30 22:19:49.861744 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:19:49.861723    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b474842685af055af4df70c9fc33183c360aa6c54b16b78f0fad2720e923420\": container with ID starting with 7b474842685af055af4df70c9fc33183c360aa6c54b16b78f0fad2720e923420 not found: ID does not exist" containerID="7b474842685af055af4df70c9fc33183c360aa6c54b16b78f0fad2720e923420"
Oct 30 22:19:49.861849 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.861751    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b474842685af055af4df70c9fc33183c360aa6c54b16b78f0fad2720e923420"} err="failed to get container status \"7b474842685af055af4df70c9fc33183c360aa6c54b16b78f0fad2720e923420\": rpc error: code = NotFound desc = could not find container \"7b474842685af055af4df70c9fc33183c360aa6c54b16b78f0fad2720e923420\": container with ID starting with 7b474842685af055af4df70c9fc33183c360aa6c54b16b78f0fad2720e923420 not found: ID does not exist"
Oct 30 22:19:49.861849 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.861768    5586 scope.go:122] "RemoveContainer" containerID="82b93ee8f4e0657f07b778d752df9415be6df278addfdfb9a327430e339434cb"
Oct 30 22:19:49.862035 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:19:49.862017    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82b93ee8f4e0657f07b778d752df9415be6df278addfdfb9a327430e339434cb\": container with ID starting with 82b93ee8f4e0657f07b778d752df9415be6df278addfdfb9a327430e339434cb not found: ID does not exist" containerID="82b93ee8f4e0657f07b778d752df9415be6df278addfdfb9a327430e339434cb"
Oct 30 22:19:49.862092 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:49.862039    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82b93ee8f4e0657f07b778d752df9415be6df278addfdfb9a327430e339434cb"} err="failed to get container status \"82b93ee8f4e0657f07b778d752df9415be6df278addfdfb9a327430e339434cb\": rpc error: code = NotFound desc = could not find container \"82b93ee8f4e0657f07b778d752df9415be6df278addfdfb9a327430e339434cb\": container with ID starting with 82b93ee8f4e0657f07b778d752df9415be6df278addfdfb9a327430e339434cb not found: ID does not exist"
Oct 30 22:19:49.881132 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:19:49.881059    5586 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26432f05_beba_4093_a139_217dac85db7e.slice/crio-b07e36371ae7b7f8edb971b0fae871e097852555584cc68d855ef17dbad64a73\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26432f05_beba_4093_a139_217dac85db7e.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 22:19:50.001651 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:50.001617    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"]
Oct 30 22:19:50.004826 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:50.004801    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-954628-openshift-default-789d9b4ccc-bpdzs"]
Oct 30 22:19:51.446401 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:51.446365    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="26432f05-beba-4093-a139-217dac85db7e" path="/var/lib/kubelet/pods/26432f05-beba-4093-a139-217dac85db7e/volumes"
Oct 30 22:19:51.822300 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:51.822198    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:56.719543 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:19:56.719500    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:01.797389 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:01.797351    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:05.949532 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:05.949493    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:06.220727 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:06.220623    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:06.595508 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:06.595408    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:11.395200 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:11.395163    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:15.433348 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.433312    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"]
Oct 30 22:20:15.433830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.433522    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp" podUID="d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf" containerName="maas-api" containerID="cri-o://5126a1904e84c2472870d8088258c4c5ac8c7d6aa1f23fa9e64306c89b5fade6" gracePeriod=30
Oct 30 22:20:15.553499 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.553469    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:15.688065 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.688009    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:20:15.796878 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.796842    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-openshift-service-ca\") pod \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") "
Oct 30 22:20:15.797086 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.796981    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-maas-api-tls\") pod \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") "
Oct 30 22:20:15.797086 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.797014    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-kube-api-access-2nl79\" (UniqueName: \"kubernetes.io/projected/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-kube-api-access-2nl79\") pod \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") "
Oct 30 22:20:15.797086 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.797068    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-metrics-certs\") pod \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\" (UID: \"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf\") "
Oct 30 22:20:15.797289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.797197    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-openshift-service-ca" pod "d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf" (UID: "d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:15.797434 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.797416    5586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-openshift-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:15.799088 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.799064    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-maas-api-tls" pod "d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf" (UID: "d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:15.799274 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.799254    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-kube-api-access-2nl79" pod "d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf" (UID: "d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf"). InnerVolumeSpecName "kube-api-access-2nl79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:15.799365 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.799347    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-metrics-certs" pod "d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf" (UID: "d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:15.898888 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.898850    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-maas-api-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:15.898888 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.898881    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nl79\" (UniqueName: \"kubernetes.io/projected/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-kube-api-access-2nl79\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:15.898888 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:15.898893    5586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf-metrics-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:16.004502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:16.004405    5586 generic.go:356] "Generic (PLEG): container finished" podID="d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf" containerID="5126a1904e84c2472870d8088258c4c5ac8c7d6aa1f23fa9e64306c89b5fade6" exitCode=0
Oct 30 22:20:16.004502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:16.004449    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"
Oct 30 22:20:16.004502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:16.004488    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp" event={"ID":"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf","Type":"ContainerDied","Data":"5126a1904e84c2472870d8088258c4c5ac8c7d6aa1f23fa9e64306c89b5fade6"}
Oct 30 22:20:16.004795 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:16.004539    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp" event={"ID":"d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf","Type":"ContainerDied","Data":"c39abb58ece5a239565daa98b508721abe9a91ccdaa9a76e32059a6ae0310c3c"}
Oct 30 22:20:16.004795 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:16.004569    5586 scope.go:122] "RemoveContainer" containerID="5126a1904e84c2472870d8088258c4c5ac8c7d6aa1f23fa9e64306c89b5fade6"
Oct 30 22:20:16.015442 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:16.015419    5586 scope.go:122] "RemoveContainer" containerID="5126a1904e84c2472870d8088258c4c5ac8c7d6aa1f23fa9e64306c89b5fade6"
Oct 30 22:20:16.015725 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:20:16.015708    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5126a1904e84c2472870d8088258c4c5ac8c7d6aa1f23fa9e64306c89b5fade6\": container with ID starting with 5126a1904e84c2472870d8088258c4c5ac8c7d6aa1f23fa9e64306c89b5fade6 not found: ID does not exist" containerID="5126a1904e84c2472870d8088258c4c5ac8c7d6aa1f23fa9e64306c89b5fade6"
Oct 30 22:20:16.015775 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:16.015734    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5126a1904e84c2472870d8088258c4c5ac8c7d6aa1f23fa9e64306c89b5fade6"} err="failed to get container status \"5126a1904e84c2472870d8088258c4c5ac8c7d6aa1f23fa9e64306c89b5fade6\": rpc error: code = NotFound desc = could not find container \"5126a1904e84c2472870d8088258c4c5ac8c7d6aa1f23fa9e64306c89b5fade6\": container with ID starting with 5126a1904e84c2472870d8088258c4c5ac8c7d6aa1f23fa9e64306c89b5fade6 not found: ID does not exist"
Oct 30 22:20:16.166952 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:16.166917    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"]
Oct 30 22:20:16.216007 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:16.215969    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-326d76-77d77654bf-z74xp"]
Oct 30 22:20:17.055373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:17.055337    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:17.307762 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:17.307659    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:17.447321 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:17.447288    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf" path="/var/lib/kubelet/pods/d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf/volumes"
Oct 30 22:20:17.480778 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:17.480735    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:18.048428 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:18.048394    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:18.361951 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:20:18.361863    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:20:20.851095 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:20.851059    5586 scope.go:122] "RemoveContainer" containerID="3ab68cda2abf99e5701587a65cca7d83389636aea4268b39544aa8f0f150c40b"
Oct 30 22:20:20.860458 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:20.860435    5586 scope.go:122] "RemoveContainer" containerID="e25500d3ff037bc91f26d97144297407911f0f8cecbb08cc97d260c6579db66b"
Oct 30 22:20:22.044068 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:22.044028    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"]
Oct 30 22:20:22.093375 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:22.093339    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-735132ea-v9llp"]
Oct 30 22:20:22.504369 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:22.504329    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:23.447390 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:23.447355    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ad3229f-17cf-4b4a-8cdc-459636c6ff7e" path="/var/lib/kubelet/pods/6ad3229f-17cf-4b4a-8cdc-459636c6ff7e/volumes"
Oct 30 22:20:23.847073 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:23.846989    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:24.185862 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:24.185829    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:24.345112 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:24.345074    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:24.542698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:24.542606    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:27.318950 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:27.318912    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:31.625847 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:31.625808    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:32.275352 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:32.275303    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:35.740494 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:35.740457    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"]
Oct 30 22:20:35.844695 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:35.844657    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"]
Oct 30 22:20:36.123962 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:36.123870    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:36.277023 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:36.276983    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:37.092651 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:37.092610    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:40.741341 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:40.741258    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94" podUID="0909fb48-36b5-4cd0-b3e6-f026073e3557" containerName="payload-processing" containerID="cri-o://2a8347ad62aee63ce027eede03a31115307414fabe483cd4f399a0c71e80df9f" gracePeriod=25
Oct 30 22:20:40.845058 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:40.844964    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg" podUID="0b0245bc-1651-4789-9944-5430e821cd76" containerName="payload-pre-processing" containerID="cri-o://90091a8e801156ff86322abe3cb638d62840cac2713f4ee9e1912e14633ab200" gracePeriod=25
Oct 30 22:20:40.867227 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:20:40.867204    5586 logging.go:55] [core] [Channel #7577 SubChannel #7579]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.123:50052", ServerName: "10.134.0.123:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.123:50052: connect: connection refused"
Oct 30 22:20:40.867227 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:20:40.867219    5586 logging.go:55] [core] [Channel #7578 SubChannel #7580]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.124:50052", ServerName: "10.134.0.124:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.124:50052: connect: connection refused"
Oct 30 22:20:40.998974 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:40.998881    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"
Oct 30 22:20:41.054531 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.054491    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0909fb48-36b5-4cd0-b3e6-f026073e3557-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0909fb48-36b5-4cd0-b3e6-f026073e3557-plugins-config-volume\") pod \"0909fb48-36b5-4cd0-b3e6-f026073e3557\" (UID: \"0909fb48-36b5-4cd0-b3e6-f026073e3557\") "
Oct 30 22:20:41.054753 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.054578    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0909fb48-36b5-4cd0-b3e6-f026073e3557-kube-api-access-jmnrb\" (UniqueName: \"kubernetes.io/projected/0909fb48-36b5-4cd0-b3e6-f026073e3557-kube-api-access-jmnrb\") pod \"0909fb48-36b5-4cd0-b3e6-f026073e3557\" (UID: \"0909fb48-36b5-4cd0-b3e6-f026073e3557\") "
Oct 30 22:20:41.055136 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.055096    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0909fb48-36b5-4cd0-b3e6-f026073e3557-plugins-config-volume" pod "0909fb48-36b5-4cd0-b3e6-f026073e3557" (UID: "0909fb48-36b5-4cd0-b3e6-f026073e3557"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:41.056923 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.056899    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0909fb48-36b5-4cd0-b3e6-f026073e3557-kube-api-access-jmnrb" pod "0909fb48-36b5-4cd0-b3e6-f026073e3557" (UID: "0909fb48-36b5-4cd0-b3e6-f026073e3557"). InnerVolumeSpecName "kube-api-access-jmnrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:41.103411 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.103380    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"]
Oct 30 22:20:41.103601 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.103578    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8" podUID="377ab9b6-ae8c-4acd-a7d7-9a8426031b26" containerName="maas-api" containerID="cri-o://7a5396c5703950f50b8ea14e3895ef5cd48ae3ed3a68cc3e1bf889b9c95f2849" gracePeriod=30
Oct 30 22:20:41.104951 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.104913    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"
Oct 30 22:20:41.155990 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.155950    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b0245bc-1651-4789-9944-5430e821cd76-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0245bc-1651-4789-9944-5430e821cd76-plugins-config-volume\") pod \"0b0245bc-1651-4789-9944-5430e821cd76\" (UID: \"0b0245bc-1651-4789-9944-5430e821cd76\") "
Oct 30 22:20:41.156199 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.156009    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b0245bc-1651-4789-9944-5430e821cd76-kube-api-access-7tntr\" (UniqueName: \"kubernetes.io/projected/0b0245bc-1651-4789-9944-5430e821cd76-kube-api-access-7tntr\") pod \"0b0245bc-1651-4789-9944-5430e821cd76\" (UID: \"0b0245bc-1651-4789-9944-5430e821cd76\") "
Oct 30 22:20:41.156346 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.156327    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0909fb48-36b5-4cd0-b3e6-f026073e3557-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:41.156439 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.156348    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmnrb\" (UniqueName: \"kubernetes.io/projected/0909fb48-36b5-4cd0-b3e6-f026073e3557-kube-api-access-jmnrb\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:41.156439 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.156344    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b0245bc-1651-4789-9944-5430e821cd76-plugins-config-volume" pod "0b0245bc-1651-4789-9944-5430e821cd76" (UID: "0b0245bc-1651-4789-9944-5430e821cd76"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:41.158408 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.158383    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b0245bc-1651-4789-9944-5430e821cd76-kube-api-access-7tntr" pod "0b0245bc-1651-4789-9944-5430e821cd76" (UID: "0b0245bc-1651-4789-9944-5430e821cd76"). InnerVolumeSpecName "kube-api-access-7tntr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:41.168488 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.168457    5586 generic.go:356] "Generic (PLEG): container finished" podID="0b0245bc-1651-4789-9944-5430e821cd76" containerID="90091a8e801156ff86322abe3cb638d62840cac2713f4ee9e1912e14633ab200" exitCode=0
Oct 30 22:20:41.168728 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.168539    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"
Oct 30 22:20:41.169074 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.168540    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg" event={"ID":"0b0245bc-1651-4789-9944-5430e821cd76","Type":"ContainerDied","Data":"90091a8e801156ff86322abe3cb638d62840cac2713f4ee9e1912e14633ab200"}
Oct 30 22:20:41.169183 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.169095    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg" event={"ID":"0b0245bc-1651-4789-9944-5430e821cd76","Type":"ContainerDied","Data":"27b877bb1b6ac1c159c54347fd8ec46e9deb5e178d2d0788e8f67c2fabd83783"}
Oct 30 22:20:41.169183 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.169131    5586 scope.go:122] "RemoveContainer" containerID="90091a8e801156ff86322abe3cb638d62840cac2713f4ee9e1912e14633ab200"
Oct 30 22:20:41.170414 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.170353    5586 generic.go:356] "Generic (PLEG): container finished" podID="0909fb48-36b5-4cd0-b3e6-f026073e3557" containerID="2a8347ad62aee63ce027eede03a31115307414fabe483cd4f399a0c71e80df9f" exitCode=0
Oct 30 22:20:41.170414 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.170389    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"
Oct 30 22:20:41.170569 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.170444    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94" event={"ID":"0909fb48-36b5-4cd0-b3e6-f026073e3557","Type":"ContainerDied","Data":"2a8347ad62aee63ce027eede03a31115307414fabe483cd4f399a0c71e80df9f"}
Oct 30 22:20:41.170569 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.170481    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94" event={"ID":"0909fb48-36b5-4cd0-b3e6-f026073e3557","Type":"ContainerDied","Data":"037ed3971e34fb40f63ef879d6ea2b5f2cefa566c7d4ba210005daa74205151a"}
Oct 30 22:20:41.212537 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.212503    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:41.217090 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.217067    5586 scope.go:122] "RemoveContainer" containerID="90091a8e801156ff86322abe3cb638d62840cac2713f4ee9e1912e14633ab200"
Oct 30 22:20:41.217456 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:20:41.217430    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90091a8e801156ff86322abe3cb638d62840cac2713f4ee9e1912e14633ab200\": container with ID starting with 90091a8e801156ff86322abe3cb638d62840cac2713f4ee9e1912e14633ab200 not found: ID does not exist" containerID="90091a8e801156ff86322abe3cb638d62840cac2713f4ee9e1912e14633ab200"
Oct 30 22:20:41.217579 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.217466    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90091a8e801156ff86322abe3cb638d62840cac2713f4ee9e1912e14633ab200"} err="failed to get container status \"90091a8e801156ff86322abe3cb638d62840cac2713f4ee9e1912e14633ab200\": rpc error: code = NotFound desc = could not find container \"90091a8e801156ff86322abe3cb638d62840cac2713f4ee9e1912e14633ab200\": container with ID starting with 90091a8e801156ff86322abe3cb638d62840cac2713f4ee9e1912e14633ab200 not found: ID does not exist"
Oct 30 22:20:41.217579 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.217488    5586 scope.go:122] "RemoveContainer" containerID="2a8347ad62aee63ce027eede03a31115307414fabe483cd4f399a0c71e80df9f"
Oct 30 22:20:41.227715 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.227695    5586 scope.go:122] "RemoveContainer" containerID="2a8347ad62aee63ce027eede03a31115307414fabe483cd4f399a0c71e80df9f"
Oct 30 22:20:41.228053 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:20:41.228029    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a8347ad62aee63ce027eede03a31115307414fabe483cd4f399a0c71e80df9f\": container with ID starting with 2a8347ad62aee63ce027eede03a31115307414fabe483cd4f399a0c71e80df9f not found: ID does not exist" containerID="2a8347ad62aee63ce027eede03a31115307414fabe483cd4f399a0c71e80df9f"
Oct 30 22:20:41.228108 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.228065    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a8347ad62aee63ce027eede03a31115307414fabe483cd4f399a0c71e80df9f"} err="failed to get container status \"2a8347ad62aee63ce027eede03a31115307414fabe483cd4f399a0c71e80df9f\": rpc error: code = NotFound desc = could not find container \"2a8347ad62aee63ce027eede03a31115307414fabe483cd4f399a0c71e80df9f\": container with ID starting with 2a8347ad62aee63ce027eede03a31115307414fabe483cd4f399a0c71e80df9f not found: ID does not exist"
Oct 30 22:20:41.257675 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.257531    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0245bc-1651-4789-9944-5430e821cd76-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:41.257675 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.257565    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tntr\" (UniqueName: \"kubernetes.io/projected/0b0245bc-1651-4789-9944-5430e821cd76-kube-api-access-7tntr\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:41.363720 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.363690    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:20:41.460063 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.460023    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-metrics-certs\" (UniqueName: \"kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-metrics-certs\") pod \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") "
Oct 30 22:20:41.460300 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.460127    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-kube-api-access-4fq2d\" (UniqueName: \"kubernetes.io/projected/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-kube-api-access-4fq2d\") pod \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") "
Oct 30 22:20:41.460300 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.460280    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-maas-api-tls\") pod \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") "
Oct 30 22:20:41.460443 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.460344    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-openshift-service-ca\") pod \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\" (UID: \"377ab9b6-ae8c-4acd-a7d7-9a8426031b26\") "
Oct 30 22:20:41.460748 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.460706    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-openshift-service-ca" pod "377ab9b6-ae8c-4acd-a7d7-9a8426031b26" (UID: "377ab9b6-ae8c-4acd-a7d7-9a8426031b26"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:41.462479 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.462450    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-kube-api-access-4fq2d" pod "377ab9b6-ae8c-4acd-a7d7-9a8426031b26" (UID: "377ab9b6-ae8c-4acd-a7d7-9a8426031b26"). InnerVolumeSpecName "kube-api-access-4fq2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:41.462606 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.462589    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-maas-api-tls" pod "377ab9b6-ae8c-4acd-a7d7-9a8426031b26" (UID: "377ab9b6-ae8c-4acd-a7d7-9a8426031b26"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:41.462767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.462750    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-metrics-certs" pod "377ab9b6-ae8c-4acd-a7d7-9a8426031b26" (UID: "377ab9b6-ae8c-4acd-a7d7-9a8426031b26"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:41.484812 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.484753    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"]
Oct 30 22:20:41.489203 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.489169    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94"]
Oct 30 22:20:41.562200 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.562091    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fq2d\" (UniqueName: \"kubernetes.io/projected/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-kube-api-access-4fq2d\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:41.562200 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.562129    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-maas-api-tls\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:41.562200 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.562139    5586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-openshift-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:41.562200 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.562149    5586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/377ab9b6-ae8c-4acd-a7d7-9a8426031b26-metrics-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:41.597107 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.597045    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"]
Oct 30 22:20:41.599318 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.599287    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg"]
Oct 30 22:20:41.866998 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.866901    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-20c7c7-b8ccd48bf-s5b94" podUID="0909fb48-36b5-4cd0-b3e6-f026073e3557" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.124:50052\" within 1s: context deadline exceeded"
Oct 30 22:20:41.867442 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:41.866907    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-20c7c7-6f67ff5655-mbghg" podUID="0b0245bc-1651-4789-9944-5430e821cd76" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.123:50052\" within 1s: context deadline exceeded"
Oct 30 22:20:42.178186 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:42.178150    5586 generic.go:356] "Generic (PLEG): container finished" podID="377ab9b6-ae8c-4acd-a7d7-9a8426031b26" containerID="7a5396c5703950f50b8ea14e3895ef5cd48ae3ed3a68cc3e1bf889b9c95f2849" exitCode=0
Oct 30 22:20:42.178425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:42.178200    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"
Oct 30 22:20:42.178425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:42.178256    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8" event={"ID":"377ab9b6-ae8c-4acd-a7d7-9a8426031b26","Type":"ContainerDied","Data":"7a5396c5703950f50b8ea14e3895ef5cd48ae3ed3a68cc3e1bf889b9c95f2849"}
Oct 30 22:20:42.178425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:42.178313    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8" event={"ID":"377ab9b6-ae8c-4acd-a7d7-9a8426031b26","Type":"ContainerDied","Data":"10dc9511052b1fd402ffc17036fd89a7f2d0d16e98a78b27ba876b3a37b20bc0"}
Oct 30 22:20:42.178425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:42.178345    5586 scope.go:122] "RemoveContainer" containerID="7a5396c5703950f50b8ea14e3895ef5cd48ae3ed3a68cc3e1bf889b9c95f2849"
Oct 30 22:20:42.189247 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:42.189215    5586 scope.go:122] "RemoveContainer" containerID="7a5396c5703950f50b8ea14e3895ef5cd48ae3ed3a68cc3e1bf889b9c95f2849"
Oct 30 22:20:42.189559 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:20:42.189539    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a5396c5703950f50b8ea14e3895ef5cd48ae3ed3a68cc3e1bf889b9c95f2849\": container with ID starting with 7a5396c5703950f50b8ea14e3895ef5cd48ae3ed3a68cc3e1bf889b9c95f2849 not found: ID does not exist" containerID="7a5396c5703950f50b8ea14e3895ef5cd48ae3ed3a68cc3e1bf889b9c95f2849"
Oct 30 22:20:42.189625 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:42.189568    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a5396c5703950f50b8ea14e3895ef5cd48ae3ed3a68cc3e1bf889b9c95f2849"} err="failed to get container status \"7a5396c5703950f50b8ea14e3895ef5cd48ae3ed3a68cc3e1bf889b9c95f2849\": rpc error: code = NotFound desc = could not find container \"7a5396c5703950f50b8ea14e3895ef5cd48ae3ed3a68cc3e1bf889b9c95f2849\": container with ID starting with 7a5396c5703950f50b8ea14e3895ef5cd48ae3ed3a68cc3e1bf889b9c95f2849 not found: ID does not exist"
Oct 30 22:20:42.340979 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:42.340935    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"]
Oct 30 22:20:42.390245 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:42.390189    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-20c7c7-7d57b5dfd8-st9g8"]
Oct 30 22:20:42.748162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:42.748126    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:43.005289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:43.005176    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:43.157264 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:43.157201    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:43.453616 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:43.453579    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0909fb48-36b5-4cd0-b3e6-f026073e3557" path="/var/lib/kubelet/pods/0909fb48-36b5-4cd0-b3e6-f026073e3557/volumes"
Oct 30 22:20:43.454004 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:43.453982    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b0245bc-1651-4789-9944-5430e821cd76" path="/var/lib/kubelet/pods/0b0245bc-1651-4789-9944-5430e821cd76/volumes"
Oct 30 22:20:43.454410 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:43.454392    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="377ab9b6-ae8c-4acd-a7d7-9a8426031b26" path="/var/lib/kubelet/pods/377ab9b6-ae8c-4acd-a7d7-9a8426031b26/volumes"
Oct 30 22:20:48.087344 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:48.087309    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:48.361955 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:20:48.361860    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:20:52.790402 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:52.790364    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:53.722052 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:53.722007    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:57.591097 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:20:57.591004    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:00.125791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:00.125707    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:00.344438 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:00.344401    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:00.443493 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:00.443455    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"]
Oct 30 22:21:00.443703 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:00.443661    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7" podUID="c0960ade-ab00-437c-84a2-2c9e702a9d06" containerName="istio-proxy" containerID="cri-o://c4b6289a1b01df413d1d74a5283cd64349cc9c78e0fcb799cafa7cf26622b42d" gracePeriod=30
Oct 30 22:21:00.502650 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:00.502610    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:01.000261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.000205    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:01.506701 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.506653    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"]
Oct 30 22:21:01.507222 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507143    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d37c6852-c3a7-4827-beb7-62b415bd6308" containerName="revoke-keys"
Oct 30 22:21:01.507222 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507158    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d37c6852-c3a7-4827-beb7-62b415bd6308" containerName="revoke-keys"
Oct 30 22:21:01.507222 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507181    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf" containerName="maas-api"
Oct 30 22:21:01.507222 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507190    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf" containerName="maas-api"
Oct 30 22:21:01.507222 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507209    5586 cpu_manager.go:414] "containerMap: removing container" podUID="0b0245bc-1651-4789-9944-5430e821cd76" containerName="payload-pre-processing"
Oct 30 22:21:01.507222 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507216    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b0245bc-1651-4789-9944-5430e821cd76" containerName="payload-pre-processing"
Oct 30 22:21:01.507526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507228    5586 cpu_manager.go:414] "containerMap: removing container" podUID="26432f05-beba-4093-a139-217dac85db7e" containerName="istio-proxy"
Oct 30 22:21:01.507526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507264    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26432f05-beba-4093-a139-217dac85db7e" containerName="istio-proxy"
Oct 30 22:21:01.507526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507280    5586 cpu_manager.go:414] "containerMap: removing container" podUID="0909fb48-36b5-4cd0-b3e6-f026073e3557" containerName="payload-processing"
Oct 30 22:21:01.507526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507285    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0909fb48-36b5-4cd0-b3e6-f026073e3557" containerName="payload-processing"
Oct 30 22:21:01.507526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507297    5586 cpu_manager.go:414] "containerMap: removing container" podUID="26432f05-beba-4093-a139-217dac85db7e" containerName="istio-proxy"
Oct 30 22:21:01.507526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507302    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26432f05-beba-4093-a139-217dac85db7e" containerName="istio-proxy"
Oct 30 22:21:01.507526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507316    5586 cpu_manager.go:414] "containerMap: removing container" podUID="377ab9b6-ae8c-4acd-a7d7-9a8426031b26" containerName="maas-api"
Oct 30 22:21:01.507526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507321    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="377ab9b6-ae8c-4acd-a7d7-9a8426031b26" containerName="maas-api"
Oct 30 22:21:01.507526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507393    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="26432f05-beba-4093-a139-217dac85db7e" containerName="istio-proxy"
Oct 30 22:21:01.507526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507407    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="377ab9b6-ae8c-4acd-a7d7-9a8426031b26" containerName="maas-api"
Oct 30 22:21:01.507526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507418    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d37c6852-c3a7-4827-beb7-62b415bd6308" containerName="revoke-keys"
Oct 30 22:21:01.507526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507426    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5aaf7bb-d2e0-49ad-99fd-5c93ee1dcadf" containerName="maas-api"
Oct 30 22:21:01.507526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507434    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b0245bc-1651-4789-9944-5430e821cd76" containerName="payload-pre-processing"
Oct 30 22:21:01.507526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.507441    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="0909fb48-36b5-4cd0-b3e6-f026073e3557" containerName="payload-processing"
Oct 30 22:21:01.510605 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.510583    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"
Oct 30 22:21:01.564252 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.564194    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9597f220-c050-4723-b139-badb6b137731-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw\" (UID: \"9597f220-c050-4723-b139-badb6b137731\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"
Oct 30 22:21:01.564473 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.564268    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4zjx\" (UniqueName: \"kubernetes.io/projected/9597f220-c050-4723-b139-badb6b137731-kube-api-access-k4zjx\") pod \"maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw\" (UID: \"9597f220-c050-4723-b139-badb6b137731\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"
Oct 30 22:21:01.564473 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.564450    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9597f220-c050-4723-b139-badb6b137731-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw\" (UID: \"9597f220-c050-4723-b139-badb6b137731\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"
Oct 30 22:21:01.564802 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.564746    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:21:01.565098 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.564884    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:21:01.614288 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.614254    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:21:01.614494 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.614292    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:21:01.616795 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.616767    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"]
Oct 30 22:21:01.665479 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.665435    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9597f220-c050-4723-b139-badb6b137731-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw\" (UID: \"9597f220-c050-4723-b139-badb6b137731\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"
Oct 30 22:21:01.665710 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.665489    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9597f220-c050-4723-b139-badb6b137731-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw\" (UID: \"9597f220-c050-4723-b139-badb6b137731\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"
Oct 30 22:21:01.665710 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.665521    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4zjx\" (UniqueName: \"kubernetes.io/projected/9597f220-c050-4723-b139-badb6b137731-kube-api-access-k4zjx\") pod \"maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw\" (UID: \"9597f220-c050-4723-b139-badb6b137731\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"
Oct 30 22:21:01.666211 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.666187    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9597f220-c050-4723-b139-badb6b137731-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw\" (UID: \"9597f220-c050-4723-b139-badb6b137731\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"
Oct 30 22:21:01.667921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.667903    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9597f220-c050-4723-b139-badb6b137731-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw\" (UID: \"9597f220-c050-4723-b139-badb6b137731\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"
Oct 30 22:21:01.722466 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.722433    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4zjx\" (UniqueName: \"kubernetes.io/projected/9597f220-c050-4723-b139-badb6b137731-kube-api-access-k4zjx\") pod \"maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw\" (UID: \"9597f220-c050-4723-b139-badb6b137731\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"
Oct 30 22:21:01.823777 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:01.823674    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"
Oct 30 22:21:02.213469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:02.213431    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"]
Oct 30 22:21:02.220530 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:21:02.220484    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9597f220_c050_4723_b139_badb6b137731.slice/crio-9e967160e2c40f7e4755d7030bf28654616e0fa0ed9ba0fe9b10a43e3b1efcce WatchSource:0}: Error finding container 9e967160e2c40f7e4755d7030bf28654616e0fa0ed9ba0fe9b10a43e3b1efcce: Status 404 returned error can't find the container with id 9e967160e2c40f7e4755d7030bf28654616e0fa0ed9ba0fe9b10a43e3b1efcce
Oct 30 22:21:02.304528 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:02.304489    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw" event={"ID":"9597f220-c050-4723-b139-badb6b137731","Type":"ContainerStarted","Data":"9e967160e2c40f7e4755d7030bf28654616e0fa0ed9ba0fe9b10a43e3b1efcce"}
Oct 30 22:21:02.500349 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:02.500270    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:03.311813 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:03.311771    5586 generic.go:356] "Generic (PLEG): container finished" podID="9597f220-c050-4723-b139-badb6b137731" containerID="e6b99867f20085f2d9cfe34e8639a2ab9b4418393600427a39afdae2b3733a14" exitCode=0
Oct 30 22:21:03.312251 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:03.311856    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw" event={"ID":"9597f220-c050-4723-b139-badb6b137731","Type":"ContainerDied","Data":"e6b99867f20085f2d9cfe34e8639a2ab9b4418393600427a39afdae2b3733a14"}
Oct 30 22:21:04.457151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:04.457119    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"
Oct 30 22:21:04.595222 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:04.595140    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9597f220-c050-4723-b139-badb6b137731-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9597f220-c050-4723-b139-badb6b137731-maas-api-openshift-ca\") pod \"9597f220-c050-4723-b139-badb6b137731\" (UID: \"9597f220-c050-4723-b139-badb6b137731\") "
Oct 30 22:21:04.595222 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:04.595196    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9597f220-c050-4723-b139-badb6b137731-kube-api-access-k4zjx\" (UniqueName: \"kubernetes.io/projected/9597f220-c050-4723-b139-badb6b137731-kube-api-access-k4zjx\") pod \"9597f220-c050-4723-b139-badb6b137731\" (UID: \"9597f220-c050-4723-b139-badb6b137731\") "
Oct 30 22:21:04.595441 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:04.595280    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9597f220-c050-4723-b139-badb6b137731-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9597f220-c050-4723-b139-badb6b137731-maas-api-xks-ca\") pod \"9597f220-c050-4723-b139-badb6b137731\" (UID: \"9597f220-c050-4723-b139-badb6b137731\") "
Oct 30 22:21:04.595658 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:04.595628    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9597f220-c050-4723-b139-badb6b137731-maas-api-openshift-ca" pod "9597f220-c050-4723-b139-badb6b137731" (UID: "9597f220-c050-4723-b139-badb6b137731"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:04.597547 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:04.597516    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9597f220-c050-4723-b139-badb6b137731-kube-api-access-k4zjx" pod "9597f220-c050-4723-b139-badb6b137731" (UID: "9597f220-c050-4723-b139-badb6b137731"). InnerVolumeSpecName "kube-api-access-k4zjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:04.597921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:04.597899    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9597f220-c050-4723-b139-badb6b137731-maas-api-xks-ca" pod "9597f220-c050-4723-b139-badb6b137731" (UID: "9597f220-c050-4723-b139-badb6b137731"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:21:04.696682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:04.696631    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4zjx\" (UniqueName: \"kubernetes.io/projected/9597f220-c050-4723-b139-badb6b137731-kube-api-access-k4zjx\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:04.696682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:04.696671    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9597f220-c050-4723-b139-badb6b137731-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:04.696682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:04.696686    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9597f220-c050-4723-b139-badb6b137731-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:05.325962 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.325924    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw" event={"ID":"9597f220-c050-4723-b139-badb6b137731","Type":"ContainerDied","Data":"9e967160e2c40f7e4755d7030bf28654616e0fa0ed9ba0fe9b10a43e3b1efcce"}
Oct 30 22:21:05.325962 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.325944    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"
Oct 30 22:21:05.326192 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.325975    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e967160e2c40f7e4755d7030bf28654616e0fa0ed9ba0fe9b10a43e3b1efcce"
Oct 30 22:21:05.698378 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.698348    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:21:05.806607 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.806564    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0960ade-ab00-437c-84a2-2c9e702a9d06-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0960ade-ab00-437c-84a2-2c9e702a9d06-istiod-ca-cert\") pod \"c0960ade-ab00-437c-84a2-2c9e702a9d06\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") "
Oct 30 22:21:05.806804 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.806658    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-envoy\") pod \"c0960ade-ab00-437c-84a2-2c9e702a9d06\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") "
Oct 30 22:21:05.806804 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.806752    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-workload-socket\") pod \"c0960ade-ab00-437c-84a2-2c9e702a9d06\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") "
Oct 30 22:21:05.806804 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.806792    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-podinfo\") pod \"c0960ade-ab00-437c-84a2-2c9e702a9d06\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") "
Oct 30 22:21:05.806992 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.806819    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-credential-socket\") pod \"c0960ade-ab00-437c-84a2-2c9e702a9d06\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") "
Oct 30 22:21:05.806992 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.806882    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-workload-certs\") pod \"c0960ade-ab00-437c-84a2-2c9e702a9d06\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") "
Oct 30 22:21:05.806992 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.806908    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0960ade-ab00-437c-84a2-2c9e702a9d06-kube-api-access-pv8k4\" (UniqueName: \"kubernetes.io/projected/c0960ade-ab00-437c-84a2-2c9e702a9d06-kube-api-access-pv8k4\") pod \"c0960ade-ab00-437c-84a2-2c9e702a9d06\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") "
Oct 30 22:21:05.807161 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.806988    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0960ade-ab00-437c-84a2-2c9e702a9d06-istiod-ca-cert" pod "c0960ade-ab00-437c-84a2-2c9e702a9d06" (UID: "c0960ade-ab00-437c-84a2-2c9e702a9d06"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:05.807161 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.807014    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-data\") pod \"c0960ade-ab00-437c-84a2-2c9e702a9d06\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") "
Oct 30 22:21:05.807161 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.807049    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-token\" (UniqueName: \"kubernetes.io/projected/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-token\") pod \"c0960ade-ab00-437c-84a2-2c9e702a9d06\" (UID: \"c0960ade-ab00-437c-84a2-2c9e702a9d06\") "
Oct 30 22:21:05.807375 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.807157    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-credential-socket" pod "c0960ade-ab00-437c-84a2-2c9e702a9d06" (UID: "c0960ade-ab00-437c-84a2-2c9e702a9d06"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:05.807375 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.807167    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-workload-certs" pod "c0960ade-ab00-437c-84a2-2c9e702a9d06" (UID: "c0960ade-ab00-437c-84a2-2c9e702a9d06"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:05.807375 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.807347    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-data" pod "c0960ade-ab00-437c-84a2-2c9e702a9d06" (UID: "c0960ade-ab00-437c-84a2-2c9e702a9d06"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:05.807539 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.807413    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-workload-socket" pod "c0960ade-ab00-437c-84a2-2c9e702a9d06" (UID: "c0960ade-ab00-437c-84a2-2c9e702a9d06"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:05.807539 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.807512    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:05.807539 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.807532    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:05.807666 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.807546    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0960ade-ab00-437c-84a2-2c9e702a9d06-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:05.807666 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.807562    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:05.807666 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.807579    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:05.809177 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.809144    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-token" pod "c0960ade-ab00-437c-84a2-2c9e702a9d06" (UID: "c0960ade-ab00-437c-84a2-2c9e702a9d06"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:05.809295 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.809187    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0960ade-ab00-437c-84a2-2c9e702a9d06-kube-api-access-pv8k4" pod "c0960ade-ab00-437c-84a2-2c9e702a9d06" (UID: "c0960ade-ab00-437c-84a2-2c9e702a9d06"). InnerVolumeSpecName "kube-api-access-pv8k4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:05.809435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.809414    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-envoy" pod "c0960ade-ab00-437c-84a2-2c9e702a9d06" (UID: "c0960ade-ab00-437c-84a2-2c9e702a9d06"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:05.809495 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.809447    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-podinfo" pod "c0960ade-ab00-437c-84a2-2c9e702a9d06" (UID: "c0960ade-ab00-437c-84a2-2c9e702a9d06"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:21:05.908754 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.908721    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:05.908754 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.908753    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:05.908968 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.908766    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv8k4\" (UniqueName: \"kubernetes.io/projected/c0960ade-ab00-437c-84a2-2c9e702a9d06-kube-api-access-pv8k4\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:05.908968 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:05.908776    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0960ade-ab00-437c-84a2-2c9e702a9d06-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:06.038559 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:06.038503    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:06.333280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:06.333150    5586 generic.go:356] "Generic (PLEG): container finished" podID="c0960ade-ab00-437c-84a2-2c9e702a9d06" containerID="c4b6289a1b01df413d1d74a5283cd64349cc9c78e0fcb799cafa7cf26622b42d" exitCode=0
Oct 30 22:21:06.333280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:06.333208    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"
Oct 30 22:21:06.333280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:06.333252    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7" event={"ID":"c0960ade-ab00-437c-84a2-2c9e702a9d06","Type":"ContainerDied","Data":"c4b6289a1b01df413d1d74a5283cd64349cc9c78e0fcb799cafa7cf26622b42d"}
Oct 30 22:21:06.333573 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:06.333315    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7" event={"ID":"c0960ade-ab00-437c-84a2-2c9e702a9d06","Type":"ContainerDied","Data":"591aaf2fc458c8f2bbe22ce36a1e657d302221fdf3fa3d23a93ab606e488af6b"}
Oct 30 22:21:06.333573 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:06.333340    5586 scope.go:122] "RemoveContainer" containerID="c4b6289a1b01df413d1d74a5283cd64349cc9c78e0fcb799cafa7cf26622b42d"
Oct 30 22:21:06.344282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:06.344217    5586 scope.go:122] "RemoveContainer" containerID="ff9499ce9242c4dc1ef67356bbfaef27d506b1be8433c43b4cdf3fef4a87332c"
Oct 30 22:21:06.353275 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:06.353225    5586 scope.go:122] "RemoveContainer" containerID="c4b6289a1b01df413d1d74a5283cd64349cc9c78e0fcb799cafa7cf26622b42d"
Oct 30 22:21:06.353591 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:21:06.353564    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4b6289a1b01df413d1d74a5283cd64349cc9c78e0fcb799cafa7cf26622b42d\": container with ID starting with c4b6289a1b01df413d1d74a5283cd64349cc9c78e0fcb799cafa7cf26622b42d not found: ID does not exist" containerID="c4b6289a1b01df413d1d74a5283cd64349cc9c78e0fcb799cafa7cf26622b42d"
Oct 30 22:21:06.353648 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:06.353604    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4b6289a1b01df413d1d74a5283cd64349cc9c78e0fcb799cafa7cf26622b42d"} err="failed to get container status \"c4b6289a1b01df413d1d74a5283cd64349cc9c78e0fcb799cafa7cf26622b42d\": rpc error: code = NotFound desc = could not find container \"c4b6289a1b01df413d1d74a5283cd64349cc9c78e0fcb799cafa7cf26622b42d\": container with ID starting with c4b6289a1b01df413d1d74a5283cd64349cc9c78e0fcb799cafa7cf26622b42d not found: ID does not exist"
Oct 30 22:21:06.353648 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:06.353632    5586 scope.go:122] "RemoveContainer" containerID="ff9499ce9242c4dc1ef67356bbfaef27d506b1be8433c43b4cdf3fef4a87332c"
Oct 30 22:21:06.353870 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:21:06.353848    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff9499ce9242c4dc1ef67356bbfaef27d506b1be8433c43b4cdf3fef4a87332c\": container with ID starting with ff9499ce9242c4dc1ef67356bbfaef27d506b1be8433c43b4cdf3fef4a87332c not found: ID does not exist" containerID="ff9499ce9242c4dc1ef67356bbfaef27d506b1be8433c43b4cdf3fef4a87332c"
Oct 30 22:21:06.353926 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:06.353882    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff9499ce9242c4dc1ef67356bbfaef27d506b1be8433c43b4cdf3fef4a87332c"} err="failed to get container status \"ff9499ce9242c4dc1ef67356bbfaef27d506b1be8433c43b4cdf3fef4a87332c\": rpc error: code = NotFound desc = could not find container \"ff9499ce9242c4dc1ef67356bbfaef27d506b1be8433c43b4cdf3fef4a87332c\": container with ID starting with ff9499ce9242c4dc1ef67356bbfaef27d506b1be8433c43b4cdf3fef4a87332c not found: ID does not exist"
Oct 30 22:21:06.495897 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:06.495858    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"]
Oct 30 22:21:06.501145 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:06.501118    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-20c7c7-openshift-default-bcfbb6c95-wmhh7"]
Oct 30 22:21:07.452281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:07.452220    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0960ade-ab00-437c-84a2-2c9e702a9d06" path="/var/lib/kubelet/pods/c0960ade-ab00-437c-84a2-2c9e702a9d06/volumes"
Oct 30 22:21:07.606857 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:07.606817    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:11.982424 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:11.982381    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"]
Oct 30 22:21:12.253686 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:12.253604    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:12.406504 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:12.406469    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:12.895471 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:12.895434    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"]
Oct 30 22:21:12.895724 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:12.895690    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4" podUID="fe4d37c0-63a4-4c62-a02b-97df821badbb" containerName="istio-proxy" containerID="cri-o://e473ada97c26beef41cc089ea5f07b72598064f0dadb79f64140be281e895298" gracePeriod=30
Oct 30 22:21:12.900015 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:12.899980    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"]
Oct 30 22:21:12.900224 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:12.900199    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm" podUID="e8c9a965-426c-4d2c-bffb-3752255ac87c" containerName="istio-proxy" containerID="cri-o://b666d0ca1fdcc4f2a9148bc21e33676687d62eb91518555616ec03ece2197cfc" gracePeriod=30
Oct 30 22:21:12.945250 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:12.945195    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"]
Oct 30 22:21:12.945500 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:12.945474    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd" podUID="edcb968b-97be-45b0-8e26-39378e683676" containerName="istio-proxy" containerID="cri-o://e7f27138cb98460c57f99bcb625ea6315ba52903bdf80005200ca8ff171edbc3" gracePeriod=30
Oct 30 22:21:12.948129 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:12.947268    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"]
Oct 30 22:21:12.948129 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:12.947507    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc" podUID="012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" containerName="istio-proxy" containerID="cri-o://fe2697f6f4d48cfff91e10b97c14d1f1d85ed3908b1063018e2bcef9f077ceef" gracePeriod=30
Oct 30 22:21:16.450965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.450931    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"]
Oct 30 22:21:16.451464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.451389    5586 cpu_manager.go:414] "containerMap: removing container" podUID="c0960ade-ab00-437c-84a2-2c9e702a9d06" containerName="istio-proxy"
Oct 30 22:21:16.451464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.451402    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0960ade-ab00-437c-84a2-2c9e702a9d06" containerName="istio-proxy"
Oct 30 22:21:16.451464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.451422    5586 cpu_manager.go:414] "containerMap: removing container" podUID="9597f220-c050-4723-b139-badb6b137731" containerName="revoke-keys"
Oct 30 22:21:16.451464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.451428    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9597f220-c050-4723-b139-badb6b137731" containerName="revoke-keys"
Oct 30 22:21:16.451601 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.451511    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0960ade-ab00-437c-84a2-2c9e702a9d06" containerName="istio-proxy"
Oct 30 22:21:16.451601 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.451521    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0960ade-ab00-437c-84a2-2c9e702a9d06" containerName="istio-proxy"
Oct 30 22:21:16.451601 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.451527    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="26432f05-beba-4093-a139-217dac85db7e" containerName="istio-proxy"
Oct 30 22:21:16.451601 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.451535    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="9597f220-c050-4723-b139-badb6b137731" containerName="revoke-keys"
Oct 30 22:21:16.454842 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.454822    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"
Oct 30 22:21:16.508535 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.508487    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-24vpd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:21:16.508718 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.508562    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd" podUID="edcb968b-97be-45b0-8e26-39378e683676" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:21:16.510699 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.510677    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:21:16.511767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.510933    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:21:16.511767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.511265    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:21:16.511767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.511484    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:21:16.513704 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.513672    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"]
Oct 30 22:21:16.619270 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.619207    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d1687120-f405-4ac3-9a97-2d9490acd9d5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-866238f7-2fghq\" (UID: \"d1687120-f405-4ac3-9a97-2d9490acd9d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"
Oct 30 22:21:16.619484 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.619337    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw4hs\" (UniqueName: \"kubernetes.io/projected/d1687120-f405-4ac3-9a97-2d9490acd9d5-kube-api-access-cw4hs\") pod \"maas-api-revoke-sub-866238f7-2fghq\" (UID: \"d1687120-f405-4ac3-9a97-2d9490acd9d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"
Oct 30 22:21:16.619484 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.619366    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d1687120-f405-4ac3-9a97-2d9490acd9d5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-866238f7-2fghq\" (UID: \"d1687120-f405-4ac3-9a97-2d9490acd9d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"
Oct 30 22:21:16.720621 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.720526    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d1687120-f405-4ac3-9a97-2d9490acd9d5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-866238f7-2fghq\" (UID: \"d1687120-f405-4ac3-9a97-2d9490acd9d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"
Oct 30 22:21:16.720621 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.720616    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw4hs\" (UniqueName: \"kubernetes.io/projected/d1687120-f405-4ac3-9a97-2d9490acd9d5-kube-api-access-cw4hs\") pod \"maas-api-revoke-sub-866238f7-2fghq\" (UID: \"d1687120-f405-4ac3-9a97-2d9490acd9d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"
Oct 30 22:21:16.720830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.720645    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d1687120-f405-4ac3-9a97-2d9490acd9d5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-866238f7-2fghq\" (UID: \"d1687120-f405-4ac3-9a97-2d9490acd9d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"
Oct 30 22:21:16.721376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.721349    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d1687120-f405-4ac3-9a97-2d9490acd9d5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-866238f7-2fghq\" (UID: \"d1687120-f405-4ac3-9a97-2d9490acd9d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"
Oct 30 22:21:16.722967 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.722944    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d1687120-f405-4ac3-9a97-2d9490acd9d5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-866238f7-2fghq\" (UID: \"d1687120-f405-4ac3-9a97-2d9490acd9d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"
Oct 30 22:21:16.777381 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.777342    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw4hs\" (UniqueName: \"kubernetes.io/projected/d1687120-f405-4ac3-9a97-2d9490acd9d5-kube-api-access-cw4hs\") pod \"maas-api-revoke-sub-866238f7-2fghq\" (UID: \"d1687120-f405-4ac3-9a97-2d9490acd9d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"
Oct 30 22:21:16.983070 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:16.982960    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b" podUID="56ed6524-65dc-4fe7-bebc-cdd266746388" containerName="payload-pre-processing" containerID="cri-o://8c4c01ae8471b03b19adaab304af8e0ebeca4bd5d9872eca3a3f2659efeee649" gracePeriod=25
Oct 30 22:21:17.068394 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:17.068355    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"
Oct 30 22:21:17.449206 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:17.449182    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"]
Oct 30 22:21:17.451991 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:21:17.451957    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1687120_f405_4ac3_9a97_2d9490acd9d5.slice/crio-12c64e9bdecd44cdb4999c7ca2a5f01f70f3968c2559b042665d18f6a04c7f93 WatchSource:0}: Error finding container 12c64e9bdecd44cdb4999c7ca2a5f01f70f3968c2559b042665d18f6a04c7f93: Status 404 returned error can't find the container with id 12c64e9bdecd44cdb4999c7ca2a5f01f70f3968c2559b042665d18f6a04c7f93
Oct 30 22:21:18.213211 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.213174    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:21:18.236661 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.236618    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-data\") pod \"e8c9a965-426c-4d2c-bffb-3752255ac87c\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") "
Oct 30 22:21:18.236904 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.236745    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-podinfo\") pod \"e8c9a965-426c-4d2c-bffb-3752255ac87c\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") "
Oct 30 22:21:18.236904 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.236829    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-workload-socket\") pod \"e8c9a965-426c-4d2c-bffb-3752255ac87c\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") "
Oct 30 22:21:18.236904 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.236884    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-token\" (UniqueName: \"kubernetes.io/projected/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-token\") pod \"e8c9a965-426c-4d2c-bffb-3752255ac87c\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") "
Oct 30 22:21:18.237123 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.236939    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8c9a965-426c-4d2c-bffb-3752255ac87c-kube-api-access-qx789\" (UniqueName: \"kubernetes.io/projected/e8c9a965-426c-4d2c-bffb-3752255ac87c-kube-api-access-qx789\") pod \"e8c9a965-426c-4d2c-bffb-3752255ac87c\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") "
Oct 30 22:21:18.237123 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.236983    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-credential-socket\") pod \"e8c9a965-426c-4d2c-bffb-3752255ac87c\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") "
Oct 30 22:21:18.237123 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.237013    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-data" pod "e8c9a965-426c-4d2c-bffb-3752255ac87c" (UID: "e8c9a965-426c-4d2c-bffb-3752255ac87c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.237123 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.237048    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-envoy\") pod \"e8c9a965-426c-4d2c-bffb-3752255ac87c\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") "
Oct 30 22:21:18.237123 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.237114    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-workload-certs\") pod \"e8c9a965-426c-4d2c-bffb-3752255ac87c\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") "
Oct 30 22:21:18.237419 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.237150    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8c9a965-426c-4d2c-bffb-3752255ac87c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c9a965-426c-4d2c-bffb-3752255ac87c-istiod-ca-cert\") pod \"e8c9a965-426c-4d2c-bffb-3752255ac87c\" (UID: \"e8c9a965-426c-4d2c-bffb-3752255ac87c\") "
Oct 30 22:21:18.237820 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.237790    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-credential-socket" pod "e8c9a965-426c-4d2c-bffb-3752255ac87c" (UID: "e8c9a965-426c-4d2c-bffb-3752255ac87c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.237912 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.237773    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-workload-socket" pod "e8c9a965-426c-4d2c-bffb-3752255ac87c" (UID: "e8c9a965-426c-4d2c-bffb-3752255ac87c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.238053 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.238030    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-workload-certs" pod "e8c9a965-426c-4d2c-bffb-3752255ac87c" (UID: "e8c9a965-426c-4d2c-bffb-3752255ac87c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.238269 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.238221    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8c9a965-426c-4d2c-bffb-3752255ac87c-istiod-ca-cert" pod "e8c9a965-426c-4d2c-bffb-3752255ac87c" (UID: "e8c9a965-426c-4d2c-bffb-3752255ac87c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:18.238408 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.238351    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.238408 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.238372    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c9a965-426c-4d2c-bffb-3752255ac87c-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.238593 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.238387    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.238593 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.238445    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.238593 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.238460    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.239860 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.239796    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-token" pod "e8c9a965-426c-4d2c-bffb-3752255ac87c" (UID: "e8c9a965-426c-4d2c-bffb-3752255ac87c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:18.240388 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.240352    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-envoy" pod "e8c9a965-426c-4d2c-bffb-3752255ac87c" (UID: "e8c9a965-426c-4d2c-bffb-3752255ac87c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.240875 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.240839    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c9a965-426c-4d2c-bffb-3752255ac87c-kube-api-access-qx789" pod "e8c9a965-426c-4d2c-bffb-3752255ac87c" (UID: "e8c9a965-426c-4d2c-bffb-3752255ac87c"). InnerVolumeSpecName "kube-api-access-qx789". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:18.241024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.241000    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-podinfo" pod "e8c9a965-426c-4d2c-bffb-3752255ac87c" (UID: "e8c9a965-426c-4d2c-bffb-3752255ac87c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:21:18.251495 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.251471    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 22:21:18.261134 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.261108    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 22:21:18.271193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.271159    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b" podUID="56ed6524-65dc-4fe7-bebc-cdd266746388" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:21:18.284500 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.284471    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:21:18.339477 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.339442    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-workload-certs\") pod \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") "
Oct 30 22:21:18.339677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.339489    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edcb968b-97be-45b0-8e26-39378e683676-kube-api-access-g2tt9\" (UniqueName: \"kubernetes.io/projected/edcb968b-97be-45b0-8e26-39378e683676-kube-api-access-g2tt9\") pod \"edcb968b-97be-45b0-8e26-39378e683676\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") "
Oct 30 22:21:18.339677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.339518    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-data\") pod \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") "
Oct 30 22:21:18.339677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.339548    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-istio-data\") pod \"edcb968b-97be-45b0-8e26-39378e683676\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") "
Oct 30 22:21:18.339677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.339594    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-podinfo\") pod \"fe4d37c0-63a4-4c62-a02b-97df821badbb\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") "
Oct 30 22:21:18.339677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.339633    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-credential-socket\") pod \"edcb968b-97be-45b0-8e26-39378e683676\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") "
Oct 30 22:21:18.339992 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.339808    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-workload-certs" pod "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" (UID: "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.339992 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.339905    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-data" pod "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" (UID: "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.339992 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.339945    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-token\" (UniqueName: \"kubernetes.io/projected/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-token\") pod \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") "
Oct 30 22:21:18.340159 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340000    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edcb968b-97be-45b0-8e26-39378e683676-istio-token\" (UniqueName: \"kubernetes.io/projected/edcb968b-97be-45b0-8e26-39378e683676-istio-token\") pod \"edcb968b-97be-45b0-8e26-39378e683676\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") "
Oct 30 22:21:18.340159 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340025    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-credential-socket" pod "edcb968b-97be-45b0-8e26-39378e683676" (UID: "edcb968b-97be-45b0-8e26-39378e683676"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.340159 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340038    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-istio-data" pod "edcb968b-97be-45b0-8e26-39378e683676" (UID: "edcb968b-97be-45b0-8e26-39378e683676"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.340159 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340070    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-kube-api-access-h799n\" (UniqueName: \"kubernetes.io/projected/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-kube-api-access-h799n\") pod \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") "
Oct 30 22:21:18.340159 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340123    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-credential-socket\") pod \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") "
Oct 30 22:21:18.340159 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340156    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-credential-socket\") pod \"fe4d37c0-63a4-4c62-a02b-97df821badbb\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") "
Oct 30 22:21:18.340526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340192    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-data\") pod \"fe4d37c0-63a4-4c62-a02b-97df821badbb\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") "
Oct 30 22:21:18.340526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340228    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-envoy\") pod \"fe4d37c0-63a4-4c62-a02b-97df821badbb\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") "
Oct 30 22:21:18.340526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340300    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-podinfo\") pod \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") "
Oct 30 22:21:18.340526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340358    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/edcb968b-97be-45b0-8e26-39378e683676-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edcb968b-97be-45b0-8e26-39378e683676-istiod-ca-cert\") pod \"edcb968b-97be-45b0-8e26-39378e683676\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") "
Oct 30 22:21:18.340526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340392    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-workload-socket\") pod \"edcb968b-97be-45b0-8e26-39378e683676\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") "
Oct 30 22:21:18.340526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340457    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-envoy\") pod \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") "
Oct 30 22:21:18.340526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340489    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-istio-envoy\") pod \"edcb968b-97be-45b0-8e26-39378e683676\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") "
Oct 30 22:21:18.340526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340519    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-workload-certs\") pod \"fe4d37c0-63a4-4c62-a02b-97df821badbb\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") "
Oct 30 22:21:18.340966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340559    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/edcb968b-97be-45b0-8e26-39378e683676-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edcb968b-97be-45b0-8e26-39378e683676-istio-podinfo\") pod \"edcb968b-97be-45b0-8e26-39378e683676\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") "
Oct 30 22:21:18.340966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340599    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-workload-certs\") pod \"edcb968b-97be-45b0-8e26-39378e683676\" (UID: \"edcb968b-97be-45b0-8e26-39378e683676\") "
Oct 30 22:21:18.340966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340651    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-credential-socket" pod "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" (UID: "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.340966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340664    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-workload-socket\") pod \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") "
Oct 30 22:21:18.340966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340713    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe4d37c0-63a4-4c62-a02b-97df821badbb-kube-api-access-zcv4v\" (UniqueName: \"kubernetes.io/projected/fe4d37c0-63a4-4c62-a02b-97df821badbb-kube-api-access-zcv4v\") pod \"fe4d37c0-63a4-4c62-a02b-97df821badbb\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") "
Oct 30 22:21:18.340966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340751    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe4d37c0-63a4-4c62-a02b-97df821badbb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4d37c0-63a4-4c62-a02b-97df821badbb-istiod-ca-cert\") pod \"fe4d37c0-63a4-4c62-a02b-97df821badbb\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") "
Oct 30 22:21:18.340966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340790    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-workload-socket\") pod \"fe4d37c0-63a4-4c62-a02b-97df821badbb\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") "
Oct 30 22:21:18.340966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340824    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-token\" (UniqueName: \"kubernetes.io/projected/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-token\") pod \"fe4d37c0-63a4-4c62-a02b-97df821badbb\" (UID: \"fe4d37c0-63a4-4c62-a02b-97df821badbb\") "
Oct 30 22:21:18.340966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340855    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istiod-ca-cert\") pod \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\" (UID: \"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac\") "
Oct 30 22:21:18.340966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340863    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edcb968b-97be-45b0-8e26-39378e683676-istiod-ca-cert" pod "edcb968b-97be-45b0-8e26-39378e683676" (UID: "edcb968b-97be-45b0-8e26-39378e683676"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:18.340966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.340878    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-workload-socket" pod "edcb968b-97be-45b0-8e26-39378e683676" (UID: "edcb968b-97be-45b0-8e26-39378e683676"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341050    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-credential-socket" pod "fe4d37c0-63a4-4c62-a02b-97df821badbb" (UID: "fe4d37c0-63a4-4c62-a02b-97df821badbb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341312    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-data" pod "fe4d37c0-63a4-4c62-a02b-97df821badbb" (UID: "fe4d37c0-63a4-4c62-a02b-97df821badbb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341402    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istiod-ca-cert" pod "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" (UID: "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341595    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341617    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341633    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341648    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx789\" (UniqueName: \"kubernetes.io/projected/e8c9a965-426c-4d2c-bffb-3752255ac87c-kube-api-access-qx789\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341662    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341676    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341692    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341705    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341721    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341736    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341751    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341765    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341780    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edcb968b-97be-45b0-8e26-39378e683676-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341795    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c9a965-426c-4d2c-bffb-3752255ac87c-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.341816    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe4d37c0-63a4-4c62-a02b-97df821badbb-istiod-ca-cert" pod "fe4d37c0-63a4-4c62-a02b-97df821badbb" (UID: "fe4d37c0-63a4-4c62-a02b-97df821badbb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.342076    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-workload-socket" pod "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" (UID: "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.342426    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-workload-certs" pod "fe4d37c0-63a4-4c62-a02b-97df821badbb" (UID: "fe4d37c0-63a4-4c62-a02b-97df821badbb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.342690    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-workload-certs" pod "edcb968b-97be-45b0-8e26-39378e683676" (UID: "edcb968b-97be-45b0-8e26-39378e683676"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.343142 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.342729    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-workload-socket" pod "fe4d37c0-63a4-4c62-a02b-97df821badbb" (UID: "fe4d37c0-63a4-4c62-a02b-97df821badbb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.344421 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.344318    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-kube-api-access-h799n" pod "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" (UID: "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac"). InnerVolumeSpecName "kube-api-access-h799n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:18.345562 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.345532    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edcb968b-97be-45b0-8e26-39378e683676-istio-token" pod "edcb968b-97be-45b0-8e26-39378e683676" (UID: "edcb968b-97be-45b0-8e26-39378e683676"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:18.345562 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.345529    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-podinfo" pod "fe4d37c0-63a4-4c62-a02b-97df821badbb" (UID: "fe4d37c0-63a4-4c62-a02b-97df821badbb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:21:18.345757 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.345557    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-token" pod "fe4d37c0-63a4-4c62-a02b-97df821badbb" (UID: "fe4d37c0-63a4-4c62-a02b-97df821badbb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:18.345757 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.345602    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-podinfo" pod "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" (UID: "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:21:18.345757 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.345692    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-envoy" pod "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" (UID: "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.345941 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.345763    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-token" pod "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" (UID: "012bb47b-2fbf-4d0e-a6dd-5e918743e5ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:18.346772 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.346726    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe4d37c0-63a4-4c62-a02b-97df821badbb-kube-api-access-zcv4v" pod "fe4d37c0-63a4-4c62-a02b-97df821badbb" (UID: "fe4d37c0-63a4-4c62-a02b-97df821badbb"). InnerVolumeSpecName "kube-api-access-zcv4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:18.347163 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.347055    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-istio-envoy" pod "edcb968b-97be-45b0-8e26-39378e683676" (UID: "edcb968b-97be-45b0-8e26-39378e683676"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.347163 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.347136    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-envoy" pod "fe4d37c0-63a4-4c62-a02b-97df821badbb" (UID: "fe4d37c0-63a4-4c62-a02b-97df821badbb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:18.347426 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.347207    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/edcb968b-97be-45b0-8e26-39378e683676-istio-podinfo" pod "edcb968b-97be-45b0-8e26-39378e683676" (UID: "edcb968b-97be-45b0-8e26-39378e683676"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:21:18.347470 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.347422    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edcb968b-97be-45b0-8e26-39378e683676-kube-api-access-g2tt9" pod "edcb968b-97be-45b0-8e26-39378e683676" (UID: "edcb968b-97be-45b0-8e26-39378e683676"). InnerVolumeSpecName "kube-api-access-g2tt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:18.365063 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:21:18.365022    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:21:18.403678 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.403638    5586 generic.go:356] "Generic (PLEG): container finished" podID="fe4d37c0-63a4-4c62-a02b-97df821badbb" containerID="e473ada97c26beef41cc089ea5f07b72598064f0dadb79f64140be281e895298" exitCode=0
Oct 30 22:21:18.403878 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.403691    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"
Oct 30 22:21:18.403878 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.403685    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4" event={"ID":"fe4d37c0-63a4-4c62-a02b-97df821badbb","Type":"ContainerDied","Data":"e473ada97c26beef41cc089ea5f07b72598064f0dadb79f64140be281e895298"}
Oct 30 22:21:18.403878 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.403817    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4" event={"ID":"fe4d37c0-63a4-4c62-a02b-97df821badbb","Type":"ContainerDied","Data":"1bdbc8db7f831aa41c2c814f7951f65c43a8355fc2081dd187672b9b3a04060b"}
Oct 30 22:21:18.403878 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.403850    5586 scope.go:122] "RemoveContainer" containerID="e473ada97c26beef41cc089ea5f07b72598064f0dadb79f64140be281e895298"
Oct 30 22:21:18.405744 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.405715    5586 generic.go:356] "Generic (PLEG): container finished" podID="d1687120-f405-4ac3-9a97-2d9490acd9d5" containerID="3c61d7c4c5c26b1f4b36d6f90dec281d246e585ea0ac51c2ba393a8e375944d5" exitCode=0
Oct 30 22:21:18.405886 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.405795    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq" event={"ID":"d1687120-f405-4ac3-9a97-2d9490acd9d5","Type":"ContainerDied","Data":"3c61d7c4c5c26b1f4b36d6f90dec281d246e585ea0ac51c2ba393a8e375944d5"}
Oct 30 22:21:18.405886 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.405851    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq" event={"ID":"d1687120-f405-4ac3-9a97-2d9490acd9d5","Type":"ContainerStarted","Data":"12c64e9bdecd44cdb4999c7ca2a5f01f70f3968c2559b042665d18f6a04c7f93"}
Oct 30 22:21:18.407539 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.407514    5586 generic.go:356] "Generic (PLEG): container finished" podID="e8c9a965-426c-4d2c-bffb-3752255ac87c" containerID="b666d0ca1fdcc4f2a9148bc21e33676687d62eb91518555616ec03ece2197cfc" exitCode=0
Oct 30 22:21:18.407674 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.407554    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"
Oct 30 22:21:18.407674 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.407581    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm" event={"ID":"e8c9a965-426c-4d2c-bffb-3752255ac87c","Type":"ContainerDied","Data":"b666d0ca1fdcc4f2a9148bc21e33676687d62eb91518555616ec03ece2197cfc"}
Oct 30 22:21:18.407674 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.407632    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm" event={"ID":"e8c9a965-426c-4d2c-bffb-3752255ac87c","Type":"ContainerDied","Data":"2a72054b89178bb7e03a8a4a233dce7baf268ad5d3769afee1e30c412caa2584"}
Oct 30 22:21:18.409721 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.409697    5586 generic.go:356] "Generic (PLEG): container finished" podID="edcb968b-97be-45b0-8e26-39378e683676" containerID="e7f27138cb98460c57f99bcb625ea6315ba52903bdf80005200ca8ff171edbc3" exitCode=0
Oct 30 22:21:18.409856 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.409725    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd" event={"ID":"edcb968b-97be-45b0-8e26-39378e683676","Type":"ContainerDied","Data":"e7f27138cb98460c57f99bcb625ea6315ba52903bdf80005200ca8ff171edbc3"}
Oct 30 22:21:18.409856 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.409772    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd" event={"ID":"edcb968b-97be-45b0-8e26-39378e683676","Type":"ContainerDied","Data":"b00f63944185b3a1b9e7f2a4b224370d76d2d40f99d6bb17e62b8db5b003faaf"}
Oct 30 22:21:18.409856 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.409775    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"
Oct 30 22:21:18.418732 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.418704    5586 scope.go:122] "RemoveContainer" containerID="e473ada97c26beef41cc089ea5f07b72598064f0dadb79f64140be281e895298"
Oct 30 22:21:18.419067 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.419036    5586 generic.go:356] "Generic (PLEG): container finished" podID="012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" containerID="fe2697f6f4d48cfff91e10b97c14d1f1d85ed3908b1063018e2bcef9f077ceef" exitCode=0
Oct 30 22:21:18.419188 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.419095    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc" event={"ID":"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac","Type":"ContainerDied","Data":"fe2697f6f4d48cfff91e10b97c14d1f1d85ed3908b1063018e2bcef9f077ceef"}
Oct 30 22:21:18.419188 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.419183    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc" event={"ID":"012bb47b-2fbf-4d0e-a6dd-5e918743e5ac","Type":"ContainerDied","Data":"9a8f823c4a6979cb62f466ba92864469b0c75676226e302dc3cf9ba8b235a91a"}
Oct 30 22:21:18.419478 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.419460    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"
Oct 30 22:21:18.421867 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:21:18.421833    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e473ada97c26beef41cc089ea5f07b72598064f0dadb79f64140be281e895298\": container with ID starting with e473ada97c26beef41cc089ea5f07b72598064f0dadb79f64140be281e895298 not found: ID does not exist" containerID="e473ada97c26beef41cc089ea5f07b72598064f0dadb79f64140be281e895298"
Oct 30 22:21:18.421993 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.421878    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e473ada97c26beef41cc089ea5f07b72598064f0dadb79f64140be281e895298"} err="failed to get container status \"e473ada97c26beef41cc089ea5f07b72598064f0dadb79f64140be281e895298\": rpc error: code = NotFound desc = could not find container \"e473ada97c26beef41cc089ea5f07b72598064f0dadb79f64140be281e895298\": container with ID starting with e473ada97c26beef41cc089ea5f07b72598064f0dadb79f64140be281e895298 not found: ID does not exist"
Oct 30 22:21:18.421993 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.421905    5586 scope.go:122] "RemoveContainer" containerID="b666d0ca1fdcc4f2a9148bc21e33676687d62eb91518555616ec03ece2197cfc"
Oct 30 22:21:18.436149 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.436125    5586 scope.go:122] "RemoveContainer" containerID="b666d0ca1fdcc4f2a9148bc21e33676687d62eb91518555616ec03ece2197cfc"
Oct 30 22:21:18.436493 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:21:18.436473    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b666d0ca1fdcc4f2a9148bc21e33676687d62eb91518555616ec03ece2197cfc\": container with ID starting with b666d0ca1fdcc4f2a9148bc21e33676687d62eb91518555616ec03ece2197cfc not found: ID does not exist" containerID="b666d0ca1fdcc4f2a9148bc21e33676687d62eb91518555616ec03ece2197cfc"
Oct 30 22:21:18.436579 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.436501    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b666d0ca1fdcc4f2a9148bc21e33676687d62eb91518555616ec03ece2197cfc"} err="failed to get container status \"b666d0ca1fdcc4f2a9148bc21e33676687d62eb91518555616ec03ece2197cfc\": rpc error: code = NotFound desc = could not find container \"b666d0ca1fdcc4f2a9148bc21e33676687d62eb91518555616ec03ece2197cfc\": container with ID starting with b666d0ca1fdcc4f2a9148bc21e33676687d62eb91518555616ec03ece2197cfc not found: ID does not exist"
Oct 30 22:21:18.436579 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.436519    5586 scope.go:122] "RemoveContainer" containerID="e7f27138cb98460c57f99bcb625ea6315ba52903bdf80005200ca8ff171edbc3"
Oct 30 22:21:18.443032 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443004    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443032 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443029    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edcb968b-97be-45b0-8e26-39378e683676-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443041    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h799n\" (UniqueName: \"kubernetes.io/projected/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-kube-api-access-h799n\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443051    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443060    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443071    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443079    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443089    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443097    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edcb968b-97be-45b0-8e26-39378e683676-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443105    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edcb968b-97be-45b0-8e26-39378e683676-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443114    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443122    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcv4v\" (UniqueName: \"kubernetes.io/projected/fe4d37c0-63a4-4c62-a02b-97df821badbb-kube-api-access-zcv4v\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443130    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4d37c0-63a4-4c62-a02b-97df821badbb-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443138    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d37c0-63a4-4c62-a02b-97df821badbb-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443146    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443156    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2tt9\" (UniqueName: \"kubernetes.io/projected/edcb968b-97be-45b0-8e26-39378e683676-kube-api-access-g2tt9\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.443209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.443164    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4d37c0-63a4-4c62-a02b-97df821badbb-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.446134 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.446111    5586 scope.go:122] "RemoveContainer" containerID="50bcd58fc15c87b9bb68c433650fda8162f3d2e646620e8d85b527ddb89743ea"
Oct 30 22:21:18.457820 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.457790    5586 scope.go:122] "RemoveContainer" containerID="e7f27138cb98460c57f99bcb625ea6315ba52903bdf80005200ca8ff171edbc3"
Oct 30 22:21:18.458190 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:21:18.458140    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7f27138cb98460c57f99bcb625ea6315ba52903bdf80005200ca8ff171edbc3\": container with ID starting with e7f27138cb98460c57f99bcb625ea6315ba52903bdf80005200ca8ff171edbc3 not found: ID does not exist" containerID="e7f27138cb98460c57f99bcb625ea6315ba52903bdf80005200ca8ff171edbc3"
Oct 30 22:21:18.458190 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.458168    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7f27138cb98460c57f99bcb625ea6315ba52903bdf80005200ca8ff171edbc3"} err="failed to get container status \"e7f27138cb98460c57f99bcb625ea6315ba52903bdf80005200ca8ff171edbc3\": rpc error: code = NotFound desc = could not find container \"e7f27138cb98460c57f99bcb625ea6315ba52903bdf80005200ca8ff171edbc3\": container with ID starting with e7f27138cb98460c57f99bcb625ea6315ba52903bdf80005200ca8ff171edbc3 not found: ID does not exist"
Oct 30 22:21:18.458190 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.458186    5586 scope.go:122] "RemoveContainer" containerID="50bcd58fc15c87b9bb68c433650fda8162f3d2e646620e8d85b527ddb89743ea"
Oct 30 22:21:18.458488 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:21:18.458473    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50bcd58fc15c87b9bb68c433650fda8162f3d2e646620e8d85b527ddb89743ea\": container with ID starting with 50bcd58fc15c87b9bb68c433650fda8162f3d2e646620e8d85b527ddb89743ea not found: ID does not exist" containerID="50bcd58fc15c87b9bb68c433650fda8162f3d2e646620e8d85b527ddb89743ea"
Oct 30 22:21:18.458551 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.458492    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50bcd58fc15c87b9bb68c433650fda8162f3d2e646620e8d85b527ddb89743ea"} err="failed to get container status \"50bcd58fc15c87b9bb68c433650fda8162f3d2e646620e8d85b527ddb89743ea\": rpc error: code = NotFound desc = could not find container \"50bcd58fc15c87b9bb68c433650fda8162f3d2e646620e8d85b527ddb89743ea\": container with ID starting with 50bcd58fc15c87b9bb68c433650fda8162f3d2e646620e8d85b527ddb89743ea not found: ID does not exist"
Oct 30 22:21:18.458551 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.458505    5586 scope.go:122] "RemoveContainer" containerID="fe2697f6f4d48cfff91e10b97c14d1f1d85ed3908b1063018e2bcef9f077ceef"
Oct 30 22:21:18.468024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.467989    5586 scope.go:122] "RemoveContainer" containerID="fe2697f6f4d48cfff91e10b97c14d1f1d85ed3908b1063018e2bcef9f077ceef"
Oct 30 22:21:18.468446 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:21:18.468418    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe2697f6f4d48cfff91e10b97c14d1f1d85ed3908b1063018e2bcef9f077ceef\": container with ID starting with fe2697f6f4d48cfff91e10b97c14d1f1d85ed3908b1063018e2bcef9f077ceef not found: ID does not exist" containerID="fe2697f6f4d48cfff91e10b97c14d1f1d85ed3908b1063018e2bcef9f077ceef"
Oct 30 22:21:18.468543 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.468453    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe2697f6f4d48cfff91e10b97c14d1f1d85ed3908b1063018e2bcef9f077ceef"} err="failed to get container status \"fe2697f6f4d48cfff91e10b97c14d1f1d85ed3908b1063018e2bcef9f077ceef\": rpc error: code = NotFound desc = could not find container \"fe2697f6f4d48cfff91e10b97c14d1f1d85ed3908b1063018e2bcef9f077ceef\": container with ID starting with fe2697f6f4d48cfff91e10b97c14d1f1d85ed3908b1063018e2bcef9f077ceef not found: ID does not exist"
Oct 30 22:21:18.843742 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.843697    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"]
Oct 30 22:21:18.849738 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.849702    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f2rlm"]
Oct 30 22:21:18.957915 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.957881    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"]
Oct 30 22:21:18.961551 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:18.961516    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-vxnxc"]
Oct 30 22:21:19.070903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.070805    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"]
Oct 30 22:21:19.075009 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.074977    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-24vpd"]
Oct 30 22:21:19.182850 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.182811    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"]
Oct 30 22:21:19.188827 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.188796    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-fwnc4"]
Oct 30 22:21:19.447511 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.447480    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" path="/var/lib/kubelet/pods/012bb47b-2fbf-4d0e-a6dd-5e918743e5ac/volumes"
Oct 30 22:21:19.448042 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.448020    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8c9a965-426c-4d2c-bffb-3752255ac87c" path="/var/lib/kubelet/pods/e8c9a965-426c-4d2c-bffb-3752255ac87c/volumes"
Oct 30 22:21:19.448543 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.448524    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="edcb968b-97be-45b0-8e26-39378e683676" path="/var/lib/kubelet/pods/edcb968b-97be-45b0-8e26-39378e683676/volumes"
Oct 30 22:21:19.449383 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.449364    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe4d37c0-63a4-4c62-a02b-97df821badbb" path="/var/lib/kubelet/pods/fe4d37c0-63a4-4c62-a02b-97df821badbb/volumes"
Oct 30 22:21:19.564454 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.564425    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"
Oct 30 22:21:19.654266 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.654205    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1687120-f405-4ac3-9a97-2d9490acd9d5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d1687120-f405-4ac3-9a97-2d9490acd9d5-maas-api-xks-ca\") pod \"d1687120-f405-4ac3-9a97-2d9490acd9d5\" (UID: \"d1687120-f405-4ac3-9a97-2d9490acd9d5\") "
Oct 30 22:21:19.654448 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.654349    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1687120-f405-4ac3-9a97-2d9490acd9d5-kube-api-access-cw4hs\" (UniqueName: \"kubernetes.io/projected/d1687120-f405-4ac3-9a97-2d9490acd9d5-kube-api-access-cw4hs\") pod \"d1687120-f405-4ac3-9a97-2d9490acd9d5\" (UID: \"d1687120-f405-4ac3-9a97-2d9490acd9d5\") "
Oct 30 22:21:19.654448 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.654387    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1687120-f405-4ac3-9a97-2d9490acd9d5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d1687120-f405-4ac3-9a97-2d9490acd9d5-maas-api-openshift-ca\") pod \"d1687120-f405-4ac3-9a97-2d9490acd9d5\" (UID: \"d1687120-f405-4ac3-9a97-2d9490acd9d5\") "
Oct 30 22:21:19.654802 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.654772    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1687120-f405-4ac3-9a97-2d9490acd9d5-maas-api-openshift-ca" pod "d1687120-f405-4ac3-9a97-2d9490acd9d5" (UID: "d1687120-f405-4ac3-9a97-2d9490acd9d5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:19.656405 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.656386    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1687120-f405-4ac3-9a97-2d9490acd9d5-maas-api-xks-ca" pod "d1687120-f405-4ac3-9a97-2d9490acd9d5" (UID: "d1687120-f405-4ac3-9a97-2d9490acd9d5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:21:19.656525 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.656428    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1687120-f405-4ac3-9a97-2d9490acd9d5-kube-api-access-cw4hs" pod "d1687120-f405-4ac3-9a97-2d9490acd9d5" (UID: "d1687120-f405-4ac3-9a97-2d9490acd9d5"). InnerVolumeSpecName "kube-api-access-cw4hs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:19.755909 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.755808    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d1687120-f405-4ac3-9a97-2d9490acd9d5-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:19.755909 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.755844    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cw4hs\" (UniqueName: \"kubernetes.io/projected/d1687120-f405-4ac3-9a97-2d9490acd9d5-kube-api-access-cw4hs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:19.755909 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:19.755855    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d1687120-f405-4ac3-9a97-2d9490acd9d5-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:20.434977 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:20.434937    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq" event={"ID":"d1687120-f405-4ac3-9a97-2d9490acd9d5","Type":"ContainerDied","Data":"12c64e9bdecd44cdb4999c7ca2a5f01f70f3968c2559b042665d18f6a04c7f93"}
Oct 30 22:21:20.435144 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:20.434989    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12c64e9bdecd44cdb4999c7ca2a5f01f70f3968c2559b042665d18f6a04c7f93"
Oct 30 22:21:20.435144 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:20.434952    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"
Oct 30 22:21:20.897945 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:20.897848    5586 scope.go:122] "RemoveContainer" containerID="93619b6ca7e1fe02d3e9d31ec6a2aa4d43a1271ee73bd04da7bd78aaac8e9609"
Oct 30 22:21:28.270548 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:28.270509    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b" podUID="56ed6524-65dc-4fe7-bebc-cdd266746388" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:21:31.908771 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:31.908725    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:32.064960 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:32.064924    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:33.436909 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:33.436872    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:33.715968 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:33.715883    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:33.987430 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:33.987348    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"]
Oct 30 22:21:34.036646 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:34.036604    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-bc009f-gn2pw"]
Oct 30 22:21:34.404597 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:34.404549    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:35.231801 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:35.231761    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:35.443556 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:35.443518    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:35.445942 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:35.445918    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9597f220-c050-4723-b139-badb6b137731" path="/var/lib/kubelet/pods/9597f220-c050-4723-b139-badb6b137731/volumes"
Oct 30 22:21:37.232330 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.232303    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"
Oct 30 22:21:37.322358 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.322272    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56ed6524-65dc-4fe7-bebc-cdd266746388-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ed6524-65dc-4fe7-bebc-cdd266746388-plugins-config-volume\") pod \"56ed6524-65dc-4fe7-bebc-cdd266746388\" (UID: \"56ed6524-65dc-4fe7-bebc-cdd266746388\") "
Oct 30 22:21:37.322358 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.322352    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56ed6524-65dc-4fe7-bebc-cdd266746388-kube-api-access-m7pvw\" (UniqueName: \"kubernetes.io/projected/56ed6524-65dc-4fe7-bebc-cdd266746388-kube-api-access-m7pvw\") pod \"56ed6524-65dc-4fe7-bebc-cdd266746388\" (UID: \"56ed6524-65dc-4fe7-bebc-cdd266746388\") "
Oct 30 22:21:37.322665 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.322643    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56ed6524-65dc-4fe7-bebc-cdd266746388-plugins-config-volume" pod "56ed6524-65dc-4fe7-bebc-cdd266746388" (UID: "56ed6524-65dc-4fe7-bebc-cdd266746388"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:37.324492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.324467    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56ed6524-65dc-4fe7-bebc-cdd266746388-kube-api-access-m7pvw" pod "56ed6524-65dc-4fe7-bebc-cdd266746388" (UID: "56ed6524-65dc-4fe7-bebc-cdd266746388"). InnerVolumeSpecName "kube-api-access-m7pvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:37.423767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.423727    5586 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ed6524-65dc-4fe7-bebc-cdd266746388-plugins-config-volume\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:37.423767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.423764    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7pvw\" (UniqueName: \"kubernetes.io/projected/56ed6524-65dc-4fe7-bebc-cdd266746388-kube-api-access-m7pvw\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:37.529033 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.528997    5586 generic.go:356] "Generic (PLEG): container finished" podID="56ed6524-65dc-4fe7-bebc-cdd266746388" containerID="8c4c01ae8471b03b19adaab304af8e0ebeca4bd5d9872eca3a3f2659efeee649" exitCode=0
Oct 30 22:21:37.529192 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.529044    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"
Oct 30 22:21:37.529192 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.529089    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b" event={"ID":"56ed6524-65dc-4fe7-bebc-cdd266746388","Type":"ContainerDied","Data":"8c4c01ae8471b03b19adaab304af8e0ebeca4bd5d9872eca3a3f2659efeee649"}
Oct 30 22:21:37.529192 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.529144    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b" event={"ID":"56ed6524-65dc-4fe7-bebc-cdd266746388","Type":"ContainerDied","Data":"6409d503bb3bdca429d88151a3fad3b87eaa39ab5c0e1073b515a631bc77d406"}
Oct 30 22:21:37.529192 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.529166    5586 scope.go:122] "RemoveContainer" containerID="8c4c01ae8471b03b19adaab304af8e0ebeca4bd5d9872eca3a3f2659efeee649"
Oct 30 22:21:37.538140 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.538112    5586 scope.go:122] "RemoveContainer" containerID="8c4c01ae8471b03b19adaab304af8e0ebeca4bd5d9872eca3a3f2659efeee649"
Oct 30 22:21:37.538455 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:21:37.538433    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c4c01ae8471b03b19adaab304af8e0ebeca4bd5d9872eca3a3f2659efeee649\": container with ID starting with 8c4c01ae8471b03b19adaab304af8e0ebeca4bd5d9872eca3a3f2659efeee649 not found: ID does not exist" containerID="8c4c01ae8471b03b19adaab304af8e0ebeca4bd5d9872eca3a3f2659efeee649"
Oct 30 22:21:37.538516 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.538463    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c4c01ae8471b03b19adaab304af8e0ebeca4bd5d9872eca3a3f2659efeee649"} err="failed to get container status \"8c4c01ae8471b03b19adaab304af8e0ebeca4bd5d9872eca3a3f2659efeee649\": rpc error: code = NotFound desc = could not find container \"8c4c01ae8471b03b19adaab304af8e0ebeca4bd5d9872eca3a3f2659efeee649\": container with ID starting with 8c4c01ae8471b03b19adaab304af8e0ebeca4bd5d9872eca3a3f2659efeee649 not found: ID does not exist"
Oct 30 22:21:37.693755 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.693719    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"]
Oct 30 22:21:37.696837 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:37.696813    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-bc009f-5fdcd96df9-9dm8b"]
Oct 30 22:21:39.446640 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:39.446607    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56ed6524-65dc-4fe7-bebc-cdd266746388" path="/var/lib/kubelet/pods/56ed6524-65dc-4fe7-bebc-cdd266746388/volumes"
Oct 30 22:21:40.361019 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:40.360982    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:40.507433 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:40.507394    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:40.719220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:40.719189    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:42.091636 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:42.091599    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:42.240299 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:42.240260    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:43.852787 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:43.852745    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:48.052689 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.052650    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7567487b8b-pnjkm"]
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053032    5586 cpu_manager.go:414] "containerMap: removing container" podUID="edcb968b-97be-45b0-8e26-39378e683676" containerName="istio-proxy"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053044    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="edcb968b-97be-45b0-8e26-39378e683676" containerName="istio-proxy"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053060    5586 cpu_manager.go:414] "containerMap: removing container" podUID="56ed6524-65dc-4fe7-bebc-cdd266746388" containerName="payload-pre-processing"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053066    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56ed6524-65dc-4fe7-bebc-cdd266746388" containerName="payload-pre-processing"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053075    5586 cpu_manager.go:414] "containerMap: removing container" podUID="fe4d37c0-63a4-4c62-a02b-97df821badbb" containerName="istio-proxy"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053079    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe4d37c0-63a4-4c62-a02b-97df821badbb" containerName="istio-proxy"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053090    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d1687120-f405-4ac3-9a97-2d9490acd9d5" containerName="revoke-keys"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053096    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1687120-f405-4ac3-9a97-2d9490acd9d5" containerName="revoke-keys"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053110    5586 cpu_manager.go:414] "containerMap: removing container" podUID="e8c9a965-426c-4d2c-bffb-3752255ac87c" containerName="istio-proxy"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053115    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8c9a965-426c-4d2c-bffb-3752255ac87c" containerName="istio-proxy"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053124    5586 cpu_manager.go:414] "containerMap: removing container" podUID="012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" containerName="istio-proxy"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053128    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" containerName="istio-proxy"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053141    5586 cpu_manager.go:414] "containerMap: removing container" podUID="c0960ade-ab00-437c-84a2-2c9e702a9d06" containerName="istio-proxy"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053146    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0960ade-ab00-437c-84a2-2c9e702a9d06" containerName="istio-proxy"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053205    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="56ed6524-65dc-4fe7-bebc-cdd266746388" containerName="payload-pre-processing"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053215    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe4d37c0-63a4-4c62-a02b-97df821badbb" containerName="istio-proxy"
Oct 30 22:21:48.053208 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053221    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1687120-f405-4ac3-9a97-2d9490acd9d5" containerName="revoke-keys"
Oct 30 22:21:48.054130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053228    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="edcb968b-97be-45b0-8e26-39378e683676" containerName="istio-proxy"
Oct 30 22:21:48.054130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053248    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8c9a965-426c-4d2c-bffb-3752255ac87c" containerName="istio-proxy"
Oct 30 22:21:48.054130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053253    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="012bb47b-2fbf-4d0e-a6dd-5e918743e5ac" containerName="istio-proxy"
Oct 30 22:21:48.054130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.053262    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="edcb968b-97be-45b0-8e26-39378e683676" containerName="istio-proxy"
Oct 30 22:21:48.056312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.056296    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:21:48.108091 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.108062    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 22:21:48.126218 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.126176    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58e89074-6d23-4557-b5d3-5845ed589a99-webhook-cert\") pod \"maas-controller-7567487b8b-pnjkm\" (UID: \"58e89074-6d23-4557-b5d3-5845ed589a99\") " pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:21:48.126412 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.126354    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhpjg\" (UniqueName: \"kubernetes.io/projected/58e89074-6d23-4557-b5d3-5845ed589a99-kube-api-access-zhpjg\") pod \"maas-controller-7567487b8b-pnjkm\" (UID: \"58e89074-6d23-4557-b5d3-5845ed589a99\") " pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:21:48.126493 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.126468    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58e89074-6d23-4557-b5d3-5845ed589a99-metrics-certs\") pod \"maas-controller-7567487b8b-pnjkm\" (UID: \"58e89074-6d23-4557-b5d3-5845ed589a99\") " pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:21:48.157801 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.157771    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 22:21:48.157801 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.157772    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7q82b\""
Oct 30 22:21:48.158033 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.157960    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 22:21:48.159899 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.159872    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7567487b8b-pnjkm"]
Oct 30 22:21:48.227624 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.227589    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhpjg\" (UniqueName: \"kubernetes.io/projected/58e89074-6d23-4557-b5d3-5845ed589a99-kube-api-access-zhpjg\") pod \"maas-controller-7567487b8b-pnjkm\" (UID: \"58e89074-6d23-4557-b5d3-5845ed589a99\") " pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:21:48.227812 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.227672    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58e89074-6d23-4557-b5d3-5845ed589a99-metrics-certs\") pod \"maas-controller-7567487b8b-pnjkm\" (UID: \"58e89074-6d23-4557-b5d3-5845ed589a99\") " pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:21:48.227812 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.227724    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58e89074-6d23-4557-b5d3-5845ed589a99-webhook-cert\") pod \"maas-controller-7567487b8b-pnjkm\" (UID: \"58e89074-6d23-4557-b5d3-5845ed589a99\") " pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:21:48.230168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.230135    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58e89074-6d23-4557-b5d3-5845ed589a99-metrics-certs\") pod \"maas-controller-7567487b8b-pnjkm\" (UID: \"58e89074-6d23-4557-b5d3-5845ed589a99\") " pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:21:48.230168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.230164    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58e89074-6d23-4557-b5d3-5845ed589a99-webhook-cert\") pod \"maas-controller-7567487b8b-pnjkm\" (UID: \"58e89074-6d23-4557-b5d3-5845ed589a99\") " pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:21:48.284877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.284847    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhpjg\" (UniqueName: \"kubernetes.io/projected/58e89074-6d23-4557-b5d3-5845ed589a99-kube-api-access-zhpjg\") pod \"maas-controller-7567487b8b-pnjkm\" (UID: \"58e89074-6d23-4557-b5d3-5845ed589a99\") " pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:21:48.361028 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:21:48.360932    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:21:48.370372 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.370350    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:21:48.747358 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.747331    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7567487b8b-pnjkm"]
Oct 30 22:21:48.749927 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:21:48.749902    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58e89074_6d23_4557_b5d3_5845ed589a99.slice/crio-f58d39f94b89c7eed72be58a1e29c896b6a96fee35442e3e890b04673148f9f7 WatchSource:0}: Error finding container f58d39f94b89c7eed72be58a1e29c896b6a96fee35442e3e890b04673148f9f7: Status 404 returned error can't find the container with id f58d39f94b89c7eed72be58a1e29c896b6a96fee35442e3e890b04673148f9f7
Oct 30 22:21:48.751194 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:48.751176    5586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:21:49.589548 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:49.589509    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:49.595662 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:49.595632    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7567487b8b-pnjkm" event={"ID":"58e89074-6d23-4557-b5d3-5845ed589a99","Type":"ContainerStarted","Data":"f58d39f94b89c7eed72be58a1e29c896b6a96fee35442e3e890b04673148f9f7"}
Oct 30 22:21:50.056466 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:50.056426    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"]
Oct 30 22:21:50.106180 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:50.106145    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-912d72be-z75rd"]
Oct 30 22:21:51.445621 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:51.445585    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ce0de86-fc7f-4cb5-8181-c20e8917f939" path="/var/lib/kubelet/pods/7ce0de86-fc7f-4cb5-8181-c20e8917f939/volumes"
Oct 30 22:21:51.608218 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:51.608178    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7567487b8b-pnjkm" event={"ID":"58e89074-6d23-4557-b5d3-5845ed589a99","Type":"ContainerStarted","Data":"3248cd1921a940ac1f14c85d19c8497f02ad61d34a6263ed874bfafb2d89d2f6"}
Oct 30 22:21:51.608426 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:51.608333    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:21:51.726165 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:21:51.726059    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-7567487b8b-pnjkm" podStartSLOduration=1.792797062 podStartE2EDuration="3.726042219s" totalImagesPullingTime="1.933245157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:51.724625192 +0000 UTC m=+2854.396529132" watchObservedRunningTime="2026-10-30 22:21:51.726042219 +0000 UTC m=+2854.397946154"
Oct 30 22:22:02.617167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:02.617127    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:22:10.908444 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:10.908405    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"]
Oct 30 22:22:10.908859 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:10.908815    5586 cpu_manager.go:414] "containerMap: removing container" podUID="edcb968b-97be-45b0-8e26-39378e683676" containerName="istio-proxy"
Oct 30 22:22:10.908859 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:10.908825    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="edcb968b-97be-45b0-8e26-39378e683676" containerName="istio-proxy"
Oct 30 22:22:10.913371 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:10.913347    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"
Oct 30 22:22:10.943627 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:10.943585    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bc23a315-9191-4cd0-8cf4-4ffb188b2109-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-87gz6\" (UID: \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"
Oct 30 22:22:10.943811 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:10.943696    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dppp6\" (UniqueName: \"kubernetes.io/projected/bc23a315-9191-4cd0-8cf4-4ffb188b2109-kube-api-access-dppp6\") pod \"maas-api-revoke-sub-85237e45-87gz6\" (UID: \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"
Oct 30 22:22:10.943811 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:10.943783    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bc23a315-9191-4cd0-8cf4-4ffb188b2109-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-87gz6\" (UID: \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"
Oct 30 22:22:10.966397 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:10.966360    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:22:11.015750 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:11.015700    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:22:11.015750 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:11.015718    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:22:11.015750 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:11.015700    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:22:11.017903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:11.017876    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"]
Oct 30 22:22:11.044702 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:11.044662    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dppp6\" (UniqueName: \"kubernetes.io/projected/bc23a315-9191-4cd0-8cf4-4ffb188b2109-kube-api-access-dppp6\") pod \"maas-api-revoke-sub-85237e45-87gz6\" (UID: \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"
Oct 30 22:22:11.044922 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:11.044759    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bc23a315-9191-4cd0-8cf4-4ffb188b2109-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-87gz6\" (UID: \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"
Oct 30 22:22:11.044922 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:11.044811    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bc23a315-9191-4cd0-8cf4-4ffb188b2109-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-87gz6\" (UID: \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"
Oct 30 22:22:11.045611 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:11.045579    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bc23a315-9191-4cd0-8cf4-4ffb188b2109-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-87gz6\" (UID: \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"
Oct 30 22:22:11.047356 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:11.047335    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bc23a315-9191-4cd0-8cf4-4ffb188b2109-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-87gz6\" (UID: \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"
Oct 30 22:22:11.101610 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:11.101569    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dppp6\" (UniqueName: \"kubernetes.io/projected/bc23a315-9191-4cd0-8cf4-4ffb188b2109-kube-api-access-dppp6\") pod \"maas-api-revoke-sub-85237e45-87gz6\" (UID: \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"
Oct 30 22:22:11.226635 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:11.226547    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"
Oct 30 22:22:11.605946 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:11.605915    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"]
Oct 30 22:22:11.608804 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:22:11.608769    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc23a315_9191_4cd0_8cf4_4ffb188b2109.slice/crio-ab7c2aa16cc861abcc469cae854407cc55bec81f3e9b34fad639d6d1d79ab64c WatchSource:0}: Error finding container ab7c2aa16cc861abcc469cae854407cc55bec81f3e9b34fad639d6d1d79ab64c: Status 404 returned error can't find the container with id ab7c2aa16cc861abcc469cae854407cc55bec81f3e9b34fad639d6d1d79ab64c
Oct 30 22:22:11.717781 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:11.717742    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6" event={"ID":"bc23a315-9191-4cd0-8cf4-4ffb188b2109","Type":"ContainerStarted","Data":"ab7c2aa16cc861abcc469cae854407cc55bec81f3e9b34fad639d6d1d79ab64c"}
Oct 30 22:22:12.725654 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:12.725614    5586 generic.go:356] "Generic (PLEG): container finished" podID="bc23a315-9191-4cd0-8cf4-4ffb188b2109" containerID="7cc4b4bf3fdea4c383f7941d52ad7855b3cc39e30174c63f20a5bc9de4ce003d" exitCode=0
Oct 30 22:22:12.726072 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:12.725695    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6" event={"ID":"bc23a315-9191-4cd0-8cf4-4ffb188b2109","Type":"ContainerDied","Data":"7cc4b4bf3fdea4c383f7941d52ad7855b3cc39e30174c63f20a5bc9de4ce003d"}
Oct 30 22:22:13.390005 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.389969    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"]
Oct 30 22:22:13.393677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.393651    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.452727 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.452697    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"]
Oct 30 22:22:13.456345 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.456322    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"]
Oct 30 22:22:13.456487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.456466    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.459846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.459823    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.468406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.468378    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.468406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.468407    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.468689 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.468426    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.468689 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.468450    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2fe270-2a33-4648-96a6-b0486a69350d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.468689 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.468588    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.468689 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.468658    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc6sr\" (UniqueName: \"kubernetes.io/projected/7e2fe270-2a33-4648-96a6-b0486a69350d-kube-api-access-mc6sr\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.468913 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.468693    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.468913 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.468710    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.468913 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.468841    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.487004 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.486970    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 22:22:13.487187 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.487015    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-xn2np\""
Oct 30 22:22:13.494871 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.494839    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"]
Oct 30 22:22:13.546097 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.546060    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"]
Oct 30 22:22:13.569709 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.569666    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.569903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.569725    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.569903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.569819    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.569903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.569867    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc6sr\" (UniqueName: \"kubernetes.io/projected/7e2fe270-2a33-4648-96a6-b0486a69350d-kube-api-access-mc6sr\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.570055 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.569922    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.570055 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.569957    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.570172 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570099    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.570172 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570144    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.570314 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570173    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.570314 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570207    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.570314 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570293    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.570487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570386    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.570487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570432    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmggh\" (UniqueName: \"kubernetes.io/projected/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-kube-api-access-lmggh\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.570487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570469    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.570612 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570491    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.570612 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570564    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.570612 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570590    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.570793 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570641    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.570793 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570703    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.570793 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570700    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.570793 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570755    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.570793 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570784    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.571058 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570829    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2fe270-2a33-4648-96a6-b0486a69350d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.571058 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570871    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.571058 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570914    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.571058 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570947    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.571058 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570974    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.571058 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.570972    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.571458 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.571070    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwnv9\" (UniqueName: \"kubernetes.io/projected/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-kube-api-access-hwnv9\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.571458 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.571352    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.571534 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.571498    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.571578 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.571538    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2fe270-2a33-4648-96a6-b0486a69350d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.572792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.572767    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.573174 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.573142    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.611130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.611087    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"]
Oct 30 22:22:13.627349 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.627316    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.627509 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.627412    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc6sr\" (UniqueName: \"kubernetes.io/projected/7e2fe270-2a33-4648-96a6-b0486a69350d-kube-api-access-mc6sr\") pod \"maas-default-gateway-openshift-default-74854d8ccf-wwv4p\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.671942 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.671903    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.672167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.671953    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.672167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.671973    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.672167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.671993    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmggh\" (UniqueName: \"kubernetes.io/projected/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-kube-api-access-lmggh\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.672167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672018    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.672167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672038    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.672167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672056    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.672167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672073    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.672167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672119    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.672167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672149    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.672682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672183    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.672682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672216    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwnv9\" (UniqueName: \"kubernetes.io/projected/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-kube-api-access-hwnv9\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.672682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672273    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.672682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672325    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.672682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672354    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.672682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672400    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.672682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672531    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.672682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672608    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.672682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672633    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.672682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672639    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.672682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672665    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.673310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672716    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.673310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672774    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.673310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.672839    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.673310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.673042    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.673310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.673057    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.673310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.673112    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.673656 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.673312    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.674996 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.674966    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.674996 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.674986    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.675191 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.675125    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.675191 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.675137    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.707285 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.707244    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:13.729746 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.729708    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmggh\" (UniqueName: \"kubernetes.io/projected/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-kube-api-access-lmggh\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.730161 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.729826    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.730161 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.729965    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-46ldm\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.730161 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.730093    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwnv9\" (UniqueName: \"kubernetes.io/projected/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-kube-api-access-hwnv9\") pod \"maas-default-gateway-openshift-default-74854d8ccf-55hwj\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.771531 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.771486    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:13.782426 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.781424    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:13.917952 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.917923    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"
Oct 30 22:22:13.975966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.975884    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc23a315-9191-4cd0-8cf4-4ffb188b2109-kube-api-access-dppp6\" (UniqueName: \"kubernetes.io/projected/bc23a315-9191-4cd0-8cf4-4ffb188b2109-kube-api-access-dppp6\") pod \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\" (UID: \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\") "
Oct 30 22:22:13.975966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.975926    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc23a315-9191-4cd0-8cf4-4ffb188b2109-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bc23a315-9191-4cd0-8cf4-4ffb188b2109-maas-api-openshift-ca\") pod \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\" (UID: \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\") "
Oct 30 22:22:13.976155 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.975990    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc23a315-9191-4cd0-8cf4-4ffb188b2109-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bc23a315-9191-4cd0-8cf4-4ffb188b2109-maas-api-xks-ca\") pod \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\" (UID: \"bc23a315-9191-4cd0-8cf4-4ffb188b2109\") "
Oct 30 22:22:13.976354 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.976326    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc23a315-9191-4cd0-8cf4-4ffb188b2109-maas-api-openshift-ca" pod "bc23a315-9191-4cd0-8cf4-4ffb188b2109" (UID: "bc23a315-9191-4cd0-8cf4-4ffb188b2109"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:22:13.978172 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.978146    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc23a315-9191-4cd0-8cf4-4ffb188b2109-maas-api-xks-ca" pod "bc23a315-9191-4cd0-8cf4-4ffb188b2109" (UID: "bc23a315-9191-4cd0-8cf4-4ffb188b2109"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:22:13.978301 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:13.978150    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc23a315-9191-4cd0-8cf4-4ffb188b2109-kube-api-access-dppp6" pod "bc23a315-9191-4cd0-8cf4-4ffb188b2109" (UID: "bc23a315-9191-4cd0-8cf4-4ffb188b2109"). InnerVolumeSpecName "kube-api-access-dppp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:22:14.076975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.076939    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dppp6\" (UniqueName: \"kubernetes.io/projected/bc23a315-9191-4cd0-8cf4-4ffb188b2109-kube-api-access-dppp6\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:14.076975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.076972    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bc23a315-9191-4cd0-8cf4-4ffb188b2109-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:14.076975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.076982    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bc23a315-9191-4cd0-8cf4-4ffb188b2109-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:14.144684 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.144648    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"]
Oct 30 22:22:14.148899 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:22:14.148867    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e2fe270_2a33_4648_96a6_b0486a69350d.slice/crio-a7dbadaa2d2b4ce5b46dac76abb28bf8e9c6ee4f67af1ecf667dd2350f137875 WatchSource:0}: Error finding container a7dbadaa2d2b4ce5b46dac76abb28bf8e9c6ee4f67af1ecf667dd2350f137875: Status 404 returned error can't find the container with id a7dbadaa2d2b4ce5b46dac76abb28bf8e9c6ee4f67af1ecf667dd2350f137875
Oct 30 22:22:14.151175 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.151143    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:14.151296 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.151210    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:14.151296 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.151253    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:14.200810 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.200780    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"]
Oct 30 22:22:14.203549 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:22:14.203518    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1dbe21e_18b9_4af9_b2c2_e36c6408543b.slice/crio-9f1288a7991605132546c81b27c9030de4e773473148b8ffbd6174a0952f6249 WatchSource:0}: Error finding container 9f1288a7991605132546c81b27c9030de4e773473148b8ffbd6174a0952f6249: Status 404 returned error can't find the container with id 9f1288a7991605132546c81b27c9030de4e773473148b8ffbd6174a0952f6249
Oct 30 22:22:14.205864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.205832    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:14.205971 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.205896    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:14.205971 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.205929    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:14.248678 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.248642    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"]
Oct 30 22:22:14.259593 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:22:14.259562    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc88a6fd6_2cac_4ad4_b523_905ed7d62bac.slice/crio-e46a24f623f8e46b43ed257c0f5faa7bd6a50609129971f9c9731248144f32ca WatchSource:0}: Error finding container e46a24f623f8e46b43ed257c0f5faa7bd6a50609129971f9c9731248144f32ca: Status 404 returned error can't find the container with id e46a24f623f8e46b43ed257c0f5faa7bd6a50609129971f9c9731248144f32ca
Oct 30 22:22:14.262376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.262337    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:14.262493 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.262440    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:14.262545 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.262496    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:14.615223 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.615180    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:14.741497 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.741462    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" event={"ID":"b1dbe21e-18b9-4af9-b2c2-e36c6408543b","Type":"ContainerStarted","Data":"6057a9d450a8eeb00e147dcb81ecfe5fc65e6650977bd8ea7c313c31330414b2"}
Oct 30 22:22:14.741937 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.741524    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" event={"ID":"b1dbe21e-18b9-4af9-b2c2-e36c6408543b","Type":"ContainerStarted","Data":"9f1288a7991605132546c81b27c9030de4e773473148b8ffbd6174a0952f6249"}
Oct 30 22:22:14.742883 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.742856    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6" event={"ID":"bc23a315-9191-4cd0-8cf4-4ffb188b2109","Type":"ContainerDied","Data":"ab7c2aa16cc861abcc469cae854407cc55bec81f3e9b34fad639d6d1d79ab64c"}
Oct 30 22:22:14.743035 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.742902    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab7c2aa16cc861abcc469cae854407cc55bec81f3e9b34fad639d6d1d79ab64c"
Oct 30 22:22:14.743112 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.743003    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"
Oct 30 22:22:14.746705 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.746672    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" event={"ID":"7e2fe270-2a33-4648-96a6-b0486a69350d","Type":"ContainerStarted","Data":"d6e6b22898f93384f1d13c9f5188a4275e40d90c8702583e090766d42e60a6ce"}
Oct 30 22:22:14.746845 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.746732    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" event={"ID":"7e2fe270-2a33-4648-96a6-b0486a69350d","Type":"ContainerStarted","Data":"a7dbadaa2d2b4ce5b46dac76abb28bf8e9c6ee4f67af1ecf667dd2350f137875"}
Oct 30 22:22:14.748636 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.748608    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" event={"ID":"c88a6fd6-2cac-4ad4-b523-905ed7d62bac","Type":"ContainerStarted","Data":"6571851fb07a096609a8a7ed53f396ee604e0f39e6b1efccdd1b65998c4418af"}
Oct 30 22:22:14.748766 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.748659    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" event={"ID":"c88a6fd6-2cac-4ad4-b523-905ed7d62bac","Type":"ContainerStarted","Data":"e46a24f623f8e46b43ed257c0f5faa7bd6a50609129971f9c9731248144f32ca"}
Oct 30 22:22:14.772650 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.772608    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:14.782749 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.782698    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:14.827094 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.826992    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:14.867420 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.867296    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" podStartSLOduration=1.867273619 podStartE2EDuration="1.867273619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:14.861764757 +0000 UTC m=+2877.533668693" watchObservedRunningTime="2026-10-30 22:22:14.867273619 +0000 UTC m=+2877.539177555"
Oct 30 22:22:14.986186 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:14.986110    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" podStartSLOduration=1.9860909740000001 podStartE2EDuration="1.986090974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:14.983903356 +0000 UTC m=+2877.655807293" watchObservedRunningTime="2026-10-30 22:22:14.986090974 +0000 UTC m=+2877.657994902"
Oct 30 22:22:15.707869 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:15.707841    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:15.709373 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:15.709333    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:22:15.709518 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:15.709405    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" podUID="7e2fe270-2a33-4648-96a6-b0486a69350d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 22:22:15.781297 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:15.780400    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.161:15021/healthz/ready\": dial tcp 10.134.0.161:15021: connect: connection refused" start-of-body=
Oct 30 22:22:15.781297 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:15.780492    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" podUID="c88a6fd6-2cac-4ad4-b523-905ed7d62bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.161:15021/healthz/ready\": dial tcp 10.134.0.161:15021: connect: connection refused"
Oct 30 22:22:15.800056 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:15.800024    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 22:22:15.800329 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:15.800292    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" podUID="b1dbe21e-18b9-4af9-b2c2-e36c6408543b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 22:22:16.708256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:16.708197    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:22:16.708463 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:16.708273    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" podUID="7e2fe270-2a33-4648-96a6-b0486a69350d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 22:22:16.778577 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:16.778530    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.161:15021/healthz/ready\": dial tcp 10.134.0.161:15021: connect: connection refused" start-of-body=
Oct 30 22:22:16.778936 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:16.778864    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" podUID="c88a6fd6-2cac-4ad4-b523-905ed7d62bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.161:15021/healthz/ready\": dial tcp 10.134.0.161:15021: connect: connection refused"
Oct 30 22:22:16.795937 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:16.795898    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 22:22:16.796651 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:16.796619    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" podUID="b1dbe21e-18b9-4af9-b2c2-e36c6408543b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 22:22:17.713272 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:17.712739    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:22:17.713272 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:17.712808    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" podUID="7e2fe270-2a33-4648-96a6-b0486a69350d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 22:22:17.793259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:17.792383    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 22:22:17.793259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:17.792455    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" podUID="b1dbe21e-18b9-4af9-b2c2-e36c6408543b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 22:22:17.793259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:17.792938    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.161:15021/healthz/ready\": dial tcp 10.134.0.161:15021: connect: connection refused" start-of-body=
Oct 30 22:22:17.793259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:17.792984    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" podUID="c88a6fd6-2cac-4ad4-b523-905ed7d62bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.161:15021/healthz/ready\": dial tcp 10.134.0.161:15021: connect: connection refused"
Oct 30 22:22:18.416592 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:22:18.416546    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:22:18.708346 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:18.708252    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:22:18.708346 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:18.708319    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" podUID="7e2fe270-2a33-4648-96a6-b0486a69350d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 22:22:18.775260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:18.772394    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.161:15021/healthz/ready\": dial tcp 10.134.0.161:15021: connect: connection refused" start-of-body=
Oct 30 22:22:18.775260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:18.772459    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" podUID="c88a6fd6-2cac-4ad4-b523-905ed7d62bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.161:15021/healthz/ready\": dial tcp 10.134.0.161:15021: connect: connection refused"
Oct 30 22:22:18.784870 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:18.782844    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 22:22:18.784870 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:18.782906    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" podUID="b1dbe21e-18b9-4af9-b2c2-e36c6408543b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 22:22:18.982351 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:18.975940    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:19.707805 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:19.707765    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:22:19.708212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:19.707826    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" podUID="7e2fe270-2a33-4648-96a6-b0486a69350d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 22:22:19.772756 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:19.772717    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.161:15021/healthz/ready\": dial tcp 10.134.0.161:15021: connect: connection refused" start-of-body=
Oct 30 22:22:19.773068 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:19.773038    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" podUID="c88a6fd6-2cac-4ad4-b523-905ed7d62bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.161:15021/healthz/ready\": dial tcp 10.134.0.161:15021: connect: connection refused"
Oct 30 22:22:19.785746 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:19.785703    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 22:22:19.786062 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:19.786019    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" podUID="b1dbe21e-18b9-4af9-b2c2-e36c6408543b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 22:22:20.707764 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:20.707720    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:22:20.707998 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:20.707777    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" podUID="7e2fe270-2a33-4648-96a6-b0486a69350d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 22:22:20.772842 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:20.772793    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.161:15021/healthz/ready\": dial tcp 10.134.0.161:15021: connect: connection refused" start-of-body=
Oct 30 22:22:20.773030 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:20.772865    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" podUID="c88a6fd6-2cac-4ad4-b523-905ed7d62bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.161:15021/healthz/ready\": dial tcp 10.134.0.161:15021: connect: connection refused"
Oct 30 22:22:20.783598 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:20.783554    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 22:22:20.783762 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:20.783617    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" podUID="b1dbe21e-18b9-4af9-b2c2-e36c6408543b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 22:22:21.708744 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:21.708701    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused" start-of-body=
Oct 30 22:22:21.709195 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:21.708771    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" podUID="7e2fe270-2a33-4648-96a6-b0486a69350d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": dial tcp 10.134.0.160:15021: connect: connection refused"
Oct 30 22:22:21.772655 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:21.772611    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.161:15021/healthz/ready\": dial tcp 10.134.0.161:15021: connect: connection refused" start-of-body=
Oct 30 22:22:21.772842 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:21.772682    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" podUID="c88a6fd6-2cac-4ad4-b523-905ed7d62bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.161:15021/healthz/ready\": dial tcp 10.134.0.161:15021: connect: connection refused"
Oct 30 22:22:21.783127 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:21.783077    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused" start-of-body=
Oct 30 22:22:21.783300 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:21.783162    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" podUID="b1dbe21e-18b9-4af9-b2c2-e36c6408543b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": dial tcp 10.134.0.162:15021: connect: connection refused"
Oct 30 22:22:23.170503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.170451    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:23.286379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.286312    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" podStartSLOduration=10.28629203 podStartE2EDuration="10.28629203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:15.1081106 +0000 UTC m=+2877.780014537" watchObservedRunningTime="2026-10-30 22:22:23.28629203 +0000 UTC m=+2885.958195966"
Oct 30 22:22:23.289727 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.289678    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7567487b8b-pnjkm"]
Oct 30 22:22:23.290044 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.289939    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-7567487b8b-pnjkm" podUID="58e89074-6d23-4557-b5d3-5845ed589a99" containerName="manager" containerID="cri-o://3248cd1921a940ac1f14c85d19c8497f02ad61d34a6263ed874bfafb2d89d2f6" gracePeriod=30
Oct 30 22:22:23.582065 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.582029    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:22:23.668108 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.668066    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58e89074-6d23-4557-b5d3-5845ed589a99-webhook-cert\" (UniqueName: \"kubernetes.io/secret/58e89074-6d23-4557-b5d3-5845ed589a99-webhook-cert\") pod \"58e89074-6d23-4557-b5d3-5845ed589a99\" (UID: \"58e89074-6d23-4557-b5d3-5845ed589a99\") "
Oct 30 22:22:23.668360 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.668169    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58e89074-6d23-4557-b5d3-5845ed589a99-metrics-certs\" (UniqueName: \"kubernetes.io/secret/58e89074-6d23-4557-b5d3-5845ed589a99-metrics-certs\") pod \"58e89074-6d23-4557-b5d3-5845ed589a99\" (UID: \"58e89074-6d23-4557-b5d3-5845ed589a99\") "
Oct 30 22:22:23.668360 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.668252    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58e89074-6d23-4557-b5d3-5845ed589a99-kube-api-access-zhpjg\" (UniqueName: \"kubernetes.io/projected/58e89074-6d23-4557-b5d3-5845ed589a99-kube-api-access-zhpjg\") pod \"58e89074-6d23-4557-b5d3-5845ed589a99\" (UID: \"58e89074-6d23-4557-b5d3-5845ed589a99\") "
Oct 30 22:22:23.670654 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.670599    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58e89074-6d23-4557-b5d3-5845ed589a99-kube-api-access-zhpjg" pod "58e89074-6d23-4557-b5d3-5845ed589a99" (UID: "58e89074-6d23-4557-b5d3-5845ed589a99"). InnerVolumeSpecName "kube-api-access-zhpjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:22:23.670964 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.670935    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58e89074-6d23-4557-b5d3-5845ed589a99-metrics-certs" pod "58e89074-6d23-4557-b5d3-5845ed589a99" (UID: "58e89074-6d23-4557-b5d3-5845ed589a99"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:22:23.671633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.671602    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58e89074-6d23-4557-b5d3-5845ed589a99-webhook-cert" pod "58e89074-6d23-4557-b5d3-5845ed589a99" (UID: "58e89074-6d23-4557-b5d3-5845ed589a99"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:22:23.710094 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.709461    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:22:23.710094 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.709531    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" podUID="7e2fe270-2a33-4648-96a6-b0486a69350d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:22:23.770143 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.770023    5586 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58e89074-6d23-4557-b5d3-5845ed589a99-webhook-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:23.770143 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.770068    5586 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58e89074-6d23-4557-b5d3-5845ed589a99-metrics-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:23.770143 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.770079    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhpjg\" (UniqueName: \"kubernetes.io/projected/58e89074-6d23-4557-b5d3-5845ed589a99-kube-api-access-zhpjg\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:23.773117 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.773047    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:22:23.773117 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.773110    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" podUID="c88a6fd6-2cac-4ad4-b523-905ed7d62bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:22:23.784436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.783900    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:22:23.784436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.783969    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" podUID="b1dbe21e-18b9-4af9-b2c2-e36c6408543b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:22:23.854717 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.854676    5586 generic.go:356] "Generic (PLEG): container finished" podID="58e89074-6d23-4557-b5d3-5845ed589a99" containerID="3248cd1921a940ac1f14c85d19c8497f02ad61d34a6263ed874bfafb2d89d2f6" exitCode=0
Oct 30 22:22:23.854717 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.854726    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7567487b8b-pnjkm"
Oct 30 22:22:23.855025 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.854777    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7567487b8b-pnjkm" event={"ID":"58e89074-6d23-4557-b5d3-5845ed589a99","Type":"ContainerDied","Data":"3248cd1921a940ac1f14c85d19c8497f02ad61d34a6263ed874bfafb2d89d2f6"}
Oct 30 22:22:23.855025 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.854844    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7567487b8b-pnjkm" event={"ID":"58e89074-6d23-4557-b5d3-5845ed589a99","Type":"ContainerDied","Data":"f58d39f94b89c7eed72be58a1e29c896b6a96fee35442e3e890b04673148f9f7"}
Oct 30 22:22:23.855025 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.854873    5586 scope.go:122] "RemoveContainer" containerID="3248cd1921a940ac1f14c85d19c8497f02ad61d34a6263ed874bfafb2d89d2f6"
Oct 30 22:22:23.869293 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.869227    5586 scope.go:122] "RemoveContainer" containerID="3248cd1921a940ac1f14c85d19c8497f02ad61d34a6263ed874bfafb2d89d2f6"
Oct 30 22:22:23.869696 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:22:23.869662    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3248cd1921a940ac1f14c85d19c8497f02ad61d34a6263ed874bfafb2d89d2f6\": container with ID starting with 3248cd1921a940ac1f14c85d19c8497f02ad61d34a6263ed874bfafb2d89d2f6 not found: ID does not exist" containerID="3248cd1921a940ac1f14c85d19c8497f02ad61d34a6263ed874bfafb2d89d2f6"
Oct 30 22:22:23.869835 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:23.869709    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3248cd1921a940ac1f14c85d19c8497f02ad61d34a6263ed874bfafb2d89d2f6"} err="failed to get container status \"3248cd1921a940ac1f14c85d19c8497f02ad61d34a6263ed874bfafb2d89d2f6\": rpc error: code = NotFound desc = could not find container \"3248cd1921a940ac1f14c85d19c8497f02ad61d34a6263ed874bfafb2d89d2f6\": container with ID starting with 3248cd1921a940ac1f14c85d19c8497f02ad61d34a6263ed874bfafb2d89d2f6 not found: ID does not exist"
Oct 30 22:22:24.019693 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:24.019654    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7567487b8b-pnjkm"]
Oct 30 22:22:24.023057 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:24.022964    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7567487b8b-pnjkm"]
Oct 30 22:22:24.753399 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:24.753354    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:22:24.754028 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:24.753991    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" podUID="7e2fe270-2a33-4648-96a6-b0486a69350d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:22:24.815398 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:24.815361    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:22:24.815590 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:24.815412    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" podUID="c88a6fd6-2cac-4ad4-b523-905ed7d62bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:22:24.840261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:24.830429    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:22:24.840261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:24.830510    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" podUID="b1dbe21e-18b9-4af9-b2c2-e36c6408543b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:22:25.477259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:25.475935    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="58e89074-6d23-4557-b5d3-5845ed589a99" path="/var/lib/kubelet/pods/58e89074-6d23-4557-b5d3-5845ed589a99/volumes"
Oct 30 22:22:25.589866 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:25.589332    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:25.798548 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:25.798451    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:22:25.799170 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:25.799133    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" podUID="7e2fe270-2a33-4648-96a6-b0486a69350d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:22:25.881758 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:25.881704    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:22:25.882112 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:25.882065    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" podUID="b1dbe21e-18b9-4af9-b2c2-e36c6408543b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:22:25.906465 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:25.906338    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:22:25.906465 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:25.906397    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" podUID="c88a6fd6-2cac-4ad4-b523-905ed7d62bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:22:26.055072 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:26.054895    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:26.056377 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:26.056347    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:26.069009 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:26.068453    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:26.069704 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:26.069676    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:26.070482 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:26.070460    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:22:26.073985 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:26.073965    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:22:26.211510 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:26.211480    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:26.211795 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:26.211775    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:26.212457 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:26.212438    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:22:28.615630 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.615545    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"]
Oct 30 22:22:28.616028 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.615965    5586 cpu_manager.go:414] "containerMap: removing container" podUID="bc23a315-9191-4cd0-8cf4-4ffb188b2109" containerName="revoke-keys"
Oct 30 22:22:28.616028 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.615976    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc23a315-9191-4cd0-8cf4-4ffb188b2109" containerName="revoke-keys"
Oct 30 22:22:28.616028 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.615989    5586 cpu_manager.go:414] "containerMap: removing container" podUID="58e89074-6d23-4557-b5d3-5845ed589a99" containerName="manager"
Oct 30 22:22:28.616028 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.615995    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58e89074-6d23-4557-b5d3-5845ed589a99" containerName="manager"
Oct 30 22:22:28.616171 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.616064    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc23a315-9191-4cd0-8cf4-4ffb188b2109" containerName="revoke-keys"
Oct 30 22:22:28.616171 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.616071    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="58e89074-6d23-4557-b5d3-5845ed589a99" containerName="manager"
Oct 30 22:22:28.621603 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.621576    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.686643 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.686589    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"]
Oct 30 22:22:28.743463 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.743425    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgmg2\" (UniqueName: \"kubernetes.io/projected/17d596a7-1c71-4494-b4f5-d9735c2204b4-kube-api-access-bgmg2\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.743463 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.743464    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.743763 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.743502    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.743763 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.743655    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.743763 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.743718    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.743900 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.743796    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.743900 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.743832    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17d596a7-1c71-4494-b4f5-d9735c2204b4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.743900 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.743861    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.744021 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.743918    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.844983 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.844940    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.845158 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.844994    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.845158 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.845023    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.845158 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.845078    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.845158 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.845118    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17d596a7-1c71-4494-b4f5-d9735c2204b4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.845473 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.845158    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.845473 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.845189    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.845473 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.845284    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgmg2\" (UniqueName: \"kubernetes.io/projected/17d596a7-1c71-4494-b4f5-d9735c2204b4-kube-api-access-bgmg2\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.845473 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.845310    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.845690 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.845513    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.845690 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.845604    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.845920 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.845893    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.846046 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.845945    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.846046 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.845972    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17d596a7-1c71-4494-b4f5-d9735c2204b4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.847485 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.847456    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.847704 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.847686    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.902379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.902298    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.902379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.902346    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgmg2\" (UniqueName: \"kubernetes.io/projected/17d596a7-1c71-4494-b4f5-d9735c2204b4-kube-api-access-bgmg2\") pod \"maas-default-gateway-openshift-default-74854d8ccf-f5gbx\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:28.935261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:28.935200    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:29.366420 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:29.366350    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"]
Oct 30 22:22:29.368729 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:22:29.368700    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17d596a7_1c71_4494_b4f5_d9735c2204b4.slice/crio-b1351638eb1f3a5e103ab5f47b33a15bc116e338f6fcf0025e8dec93225eb5a2 WatchSource:0}: Error finding container b1351638eb1f3a5e103ab5f47b33a15bc116e338f6fcf0025e8dec93225eb5a2: Status 404 returned error can't find the container with id b1351638eb1f3a5e103ab5f47b33a15bc116e338f6fcf0025e8dec93225eb5a2
Oct 30 22:22:29.370951 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:29.370917    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:29.371036 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:29.370987    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:29.371036 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:29.371018    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:22:29.949505 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:29.949467    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx" event={"ID":"17d596a7-1c71-4494-b4f5-d9735c2204b4","Type":"ContainerStarted","Data":"8ca8d2b203e0f220c49347b6a836a883291dea182656ecafbd70bb611b35c6cd"}
Oct 30 22:22:29.949943 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:29.949532    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx" event={"ID":"17d596a7-1c71-4494-b4f5-d9735c2204b4","Type":"ContainerStarted","Data":"b1351638eb1f3a5e103ab5f47b33a15bc116e338f6fcf0025e8dec93225eb5a2"}
Oct 30 22:22:30.116536 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:30.116481    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx" podStartSLOduration=2.116467896 podStartE2EDuration="2.116467896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:30.114340435 +0000 UTC m=+2892.786244367" watchObservedRunningTime="2026-10-30 22:22:30.116467896 +0000 UTC m=+2892.788371830"
Oct 30 22:22:30.945267 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:30.942916    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:30.945267 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:30.944828    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-f5gbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused" start-of-body=
Oct 30 22:22:30.945267 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:30.944901    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx" podUID="17d596a7-1c71-4494-b4f5-d9735c2204b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused"
Oct 30 22:22:31.946262 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:31.936722    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-f5gbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused" start-of-body=
Oct 30 22:22:31.946262 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:31.936790    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx" podUID="17d596a7-1c71-4494-b4f5-d9735c2204b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused"
Oct 30 22:22:32.301217 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:32.301104    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:32.945259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:32.941713    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-f5gbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused" start-of-body=
Oct 30 22:22:32.945259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:32.941779    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx" podUID="17d596a7-1c71-4494-b4f5-d9735c2204b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused"
Oct 30 22:22:33.941260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:33.940677    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-f5gbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused" start-of-body=
Oct 30 22:22:33.941260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:33.940742    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx" podUID="17d596a7-1c71-4494-b4f5-d9735c2204b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused"
Oct 30 22:22:34.937716 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:34.937673    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-f5gbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused" start-of-body=
Oct 30 22:22:34.938043 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:34.938013    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx" podUID="17d596a7-1c71-4494-b4f5-d9735c2204b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused"
Oct 30 22:22:35.936730 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:35.936685    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-f5gbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused" start-of-body=
Oct 30 22:22:35.937146 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:35.936742    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx" podUID="17d596a7-1c71-4494-b4f5-d9735c2204b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.163:15021/healthz/ready\": dial tcp 10.134.0.163:15021: connect: connection refused"
Oct 30 22:22:37.662850 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:37.662799    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:37.663577 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:37.663546    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:37.664381 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:37.664360    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:22:46.720410 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:46.720368    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:46.997357 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:22:46.997262    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:48.361769 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:22:48.361730    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:23:10.968360 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:10.968320    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:11.121620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:11.121577    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:12.965258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:12.956488    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:13.112257 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:13.111830    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:18.361833 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:23:18.361794    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:23:20.977852 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:20.977819    5586 scope.go:122] "RemoveContainer" containerID="e03a5b9bc52919e5d140ab8e598d2e06eb5820d542c188031d3ebb275463f7d6"
Oct 30 22:23:21.766594 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:21.766551    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal" podUID="6ea3e08f52bca920602c76ad35bcaf69" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:23:29.664699 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.664657    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"]
Oct 30 22:23:29.669728 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.669709    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"
Oct 30 22:23:29.709024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.708981    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5sm7\" (UniqueName: \"kubernetes.io/projected/fa31c3d5-99bf-4fea-9601-f643a0c690d5-kube-api-access-f5sm7\") pod \"maas-api-revoke-sub-c1e271a9-2zmnm\" (UID: \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"
Oct 30 22:23:29.709213 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.709098    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa31c3d5-99bf-4fea-9601-f643a0c690d5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-2zmnm\" (UID: \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"
Oct 30 22:23:29.709213 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.709166    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa31c3d5-99bf-4fea-9601-f643a0c690d5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-2zmnm\" (UID: \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"
Oct 30 22:23:29.723162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.723124    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:23:29.723162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.723128    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:23:29.772410 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.772368    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:23:29.772410 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.772376    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:23:29.774507 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.774484    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"]
Oct 30 22:23:29.810025 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.809969    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5sm7\" (UniqueName: \"kubernetes.io/projected/fa31c3d5-99bf-4fea-9601-f643a0c690d5-kube-api-access-f5sm7\") pod \"maas-api-revoke-sub-c1e271a9-2zmnm\" (UID: \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"
Oct 30 22:23:29.810294 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.810054    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa31c3d5-99bf-4fea-9601-f643a0c690d5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-2zmnm\" (UID: \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"
Oct 30 22:23:29.810294 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.810084    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa31c3d5-99bf-4fea-9601-f643a0c690d5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-2zmnm\" (UID: \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"
Oct 30 22:23:29.810792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.810769    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa31c3d5-99bf-4fea-9601-f643a0c690d5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-2zmnm\" (UID: \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"
Oct 30 22:23:29.812483 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.812457    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa31c3d5-99bf-4fea-9601-f643a0c690d5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-2zmnm\" (UID: \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"
Oct 30 22:23:29.868541 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.868504    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5sm7\" (UniqueName: \"kubernetes.io/projected/fa31c3d5-99bf-4fea-9601-f643a0c690d5-kube-api-access-f5sm7\") pod \"maas-api-revoke-sub-c1e271a9-2zmnm\" (UID: \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"
Oct 30 22:23:29.983394 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:29.983302    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"
Oct 30 22:23:30.360278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:30.360245    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"]
Oct 30 22:23:30.362675 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:23:30.362614    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa31c3d5_99bf_4fea_9601_f643a0c690d5.slice/crio-6f80606c77b398b78fb8fc5b103fb6a291385aacc3e515a48488ec93a89f21c3 WatchSource:0}: Error finding container 6f80606c77b398b78fb8fc5b103fb6a291385aacc3e515a48488ec93a89f21c3: Status 404 returned error can't find the container with id 6f80606c77b398b78fb8fc5b103fb6a291385aacc3e515a48488ec93a89f21c3
Oct 30 22:23:31.325565 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:31.325530    5586 generic.go:356] "Generic (PLEG): container finished" podID="fa31c3d5-99bf-4fea-9601-f643a0c690d5" containerID="814c6d406f4ab8e12b465a255fd27e971b4d31985a50891a90c4c9966a223a84" exitCode=0
Oct 30 22:23:31.325979 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:31.325621    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm" event={"ID":"fa31c3d5-99bf-4fea-9601-f643a0c690d5","Type":"ContainerDied","Data":"814c6d406f4ab8e12b465a255fd27e971b4d31985a50891a90c4c9966a223a84"}
Oct 30 22:23:31.325979 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:31.325679    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm" event={"ID":"fa31c3d5-99bf-4fea-9601-f643a0c690d5","Type":"ContainerStarted","Data":"6f80606c77b398b78fb8fc5b103fb6a291385aacc3e515a48488ec93a89f21c3"}
Oct 30 22:23:32.471696 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:32.471670    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"
Oct 30 22:23:32.537108 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:32.537072    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa31c3d5-99bf-4fea-9601-f643a0c690d5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa31c3d5-99bf-4fea-9601-f643a0c690d5-maas-api-openshift-ca\") pod \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\" (UID: \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\") "
Oct 30 22:23:32.537336 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:32.537123    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa31c3d5-99bf-4fea-9601-f643a0c690d5-kube-api-access-f5sm7\" (UniqueName: \"kubernetes.io/projected/fa31c3d5-99bf-4fea-9601-f643a0c690d5-kube-api-access-f5sm7\") pod \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\" (UID: \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\") "
Oct 30 22:23:32.537336 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:32.537148    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa31c3d5-99bf-4fea-9601-f643a0c690d5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa31c3d5-99bf-4fea-9601-f643a0c690d5-maas-api-xks-ca\") pod \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\" (UID: \"fa31c3d5-99bf-4fea-9601-f643a0c690d5\") "
Oct 30 22:23:32.537645 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:32.537618    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa31c3d5-99bf-4fea-9601-f643a0c690d5-maas-api-openshift-ca" pod "fa31c3d5-99bf-4fea-9601-f643a0c690d5" (UID: "fa31c3d5-99bf-4fea-9601-f643a0c690d5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:23:32.539382 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:32.539359    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa31c3d5-99bf-4fea-9601-f643a0c690d5-maas-api-xks-ca" pod "fa31c3d5-99bf-4fea-9601-f643a0c690d5" (UID: "fa31c3d5-99bf-4fea-9601-f643a0c690d5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:23:32.539484 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:32.539398    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa31c3d5-99bf-4fea-9601-f643a0c690d5-kube-api-access-f5sm7" pod "fa31c3d5-99bf-4fea-9601-f643a0c690d5" (UID: "fa31c3d5-99bf-4fea-9601-f643a0c690d5"). InnerVolumeSpecName "kube-api-access-f5sm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:32.638399 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:32.638312    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa31c3d5-99bf-4fea-9601-f643a0c690d5-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:32.638399 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:32.638344    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5sm7\" (UniqueName: \"kubernetes.io/projected/fa31c3d5-99bf-4fea-9601-f643a0c690d5-kube-api-access-f5sm7\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:32.638399 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:32.638355    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa31c3d5-99bf-4fea-9601-f643a0c690d5-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:23:32.646488 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:32.646452    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:33.338990 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:33.338959    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"
Oct 30 22:23:33.339181 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:33.338992    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm" event={"ID":"fa31c3d5-99bf-4fea-9601-f643a0c690d5","Type":"ContainerDied","Data":"6f80606c77b398b78fb8fc5b103fb6a291385aacc3e515a48488ec93a89f21c3"}
Oct 30 22:23:33.339181 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:33.339042    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f80606c77b398b78fb8fc5b103fb6a291385aacc3e515a48488ec93a89f21c3"
Oct 30 22:23:37.101213 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:37.101164    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:37.256841 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:37.256781    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:48.548697 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:23:48.548634    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:23:58.314773 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.314734    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"]
Oct 30 22:23:58.315179 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.315165    5586 cpu_manager.go:414] "containerMap: removing container" podUID="fa31c3d5-99bf-4fea-9601-f643a0c690d5" containerName="revoke-keys"
Oct 30 22:23:58.315228 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.315180    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa31c3d5-99bf-4fea-9601-f643a0c690d5" containerName="revoke-keys"
Oct 30 22:23:58.315228 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.315251    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa31c3d5-99bf-4fea-9601-f643a0c690d5" containerName="revoke-keys"
Oct 30 22:23:58.318258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.318242    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"
Oct 30 22:23:58.421514 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.421439    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:23:58.421689 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.421556    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:23:58.421689 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.421586    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:23:58.421772 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.421721    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:23:58.424040 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.424022    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"]
Oct 30 22:23:58.449485 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.449459    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ad30631-2b97-4c5b-b220-de754ab095ba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-9l85z\" (UID: \"7ad30631-2b97-4c5b-b220-de754ab095ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"
Oct 30 22:23:58.449606 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.449577    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ad30631-2b97-4c5b-b220-de754ab095ba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-9l85z\" (UID: \"7ad30631-2b97-4c5b-b220-de754ab095ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"
Oct 30 22:23:58.449649 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.449616    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-242hz\" (UniqueName: \"kubernetes.io/projected/7ad30631-2b97-4c5b-b220-de754ab095ba-kube-api-access-242hz\") pod \"maas-api-revoke-sub-2e1f5b5f-9l85z\" (UID: \"7ad30631-2b97-4c5b-b220-de754ab095ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"
Oct 30 22:23:58.550762 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.550721    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ad30631-2b97-4c5b-b220-de754ab095ba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-9l85z\" (UID: \"7ad30631-2b97-4c5b-b220-de754ab095ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"
Oct 30 22:23:58.550949 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.550858    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ad30631-2b97-4c5b-b220-de754ab095ba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-9l85z\" (UID: \"7ad30631-2b97-4c5b-b220-de754ab095ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"
Oct 30 22:23:58.550949 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.550891    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-242hz\" (UniqueName: \"kubernetes.io/projected/7ad30631-2b97-4c5b-b220-de754ab095ba-kube-api-access-242hz\") pod \"maas-api-revoke-sub-2e1f5b5f-9l85z\" (UID: \"7ad30631-2b97-4c5b-b220-de754ab095ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"
Oct 30 22:23:58.551522 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.551499    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ad30631-2b97-4c5b-b220-de754ab095ba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-9l85z\" (UID: \"7ad30631-2b97-4c5b-b220-de754ab095ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"
Oct 30 22:23:58.553218 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.553199    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ad30631-2b97-4c5b-b220-de754ab095ba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-9l85z\" (UID: \"7ad30631-2b97-4c5b-b220-de754ab095ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"
Oct 30 22:23:58.608137 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.608050    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-242hz\" (UniqueName: \"kubernetes.io/projected/7ad30631-2b97-4c5b-b220-de754ab095ba-kube-api-access-242hz\") pod \"maas-api-revoke-sub-2e1f5b5f-9l85z\" (UID: \"7ad30631-2b97-4c5b-b220-de754ab095ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"
Oct 30 22:23:58.630917 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:58.630883    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"
Oct 30 22:23:59.006301 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:59.006275    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"]
Oct 30 22:23:59.008839 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:23:59.008810    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad30631_2b97_4c5b_b220_de754ab095ba.slice/crio-90d7175a118587232c5b75ddf8751f1622686085f270629bbde34b6302796332 WatchSource:0}: Error finding container 90d7175a118587232c5b75ddf8751f1622686085f270629bbde34b6302796332: Status 404 returned error can't find the container with id 90d7175a118587232c5b75ddf8751f1622686085f270629bbde34b6302796332
Oct 30 22:23:59.514126 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:59.514080    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z" event={"ID":"7ad30631-2b97-4c5b-b220-de754ab095ba","Type":"ContainerStarted","Data":"5f01beabeb623de8089d685a12e0b91521eda0cb3e5ec09cd3ff88a0471dd5bf"}
Oct 30 22:23:59.514585 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:59.514144    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z" event={"ID":"7ad30631-2b97-4c5b-b220-de754ab095ba","Type":"ContainerStarted","Data":"90d7175a118587232c5b75ddf8751f1622686085f270629bbde34b6302796332"}
Oct 30 22:23:59.630341 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:23:59.630216    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z" podStartSLOduration=1.630202959 podStartE2EDuration="1.630202959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:23:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:23:59.627907634 +0000 UTC m=+2982.299811568" watchObservedRunningTime="2026-10-30 22:23:59.630202959 +0000 UTC m=+2982.302106937"
Oct 30 22:24:00.520950 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:00.520914    5586 generic.go:356] "Generic (PLEG): container finished" podID="7ad30631-2b97-4c5b-b220-de754ab095ba" containerID="5f01beabeb623de8089d685a12e0b91521eda0cb3e5ec09cd3ff88a0471dd5bf" exitCode=0
Oct 30 22:24:00.521407 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:00.521008    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z" event={"ID":"7ad30631-2b97-4c5b-b220-de754ab095ba","Type":"ContainerDied","Data":"5f01beabeb623de8089d685a12e0b91521eda0cb3e5ec09cd3ff88a0471dd5bf"}
Oct 30 22:24:01.676562 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:01.676529    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"
Oct 30 22:24:01.784163 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:01.784071    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ad30631-2b97-4c5b-b220-de754ab095ba-kube-api-access-242hz\" (UniqueName: \"kubernetes.io/projected/7ad30631-2b97-4c5b-b220-de754ab095ba-kube-api-access-242hz\") pod \"7ad30631-2b97-4c5b-b220-de754ab095ba\" (UID: \"7ad30631-2b97-4c5b-b220-de754ab095ba\") "
Oct 30 22:24:01.784356 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:01.784173    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ad30631-2b97-4c5b-b220-de754ab095ba-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ad30631-2b97-4c5b-b220-de754ab095ba-maas-api-xks-ca\") pod \"7ad30631-2b97-4c5b-b220-de754ab095ba\" (UID: \"7ad30631-2b97-4c5b-b220-de754ab095ba\") "
Oct 30 22:24:01.784356 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:01.784203    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ad30631-2b97-4c5b-b220-de754ab095ba-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ad30631-2b97-4c5b-b220-de754ab095ba-maas-api-openshift-ca\") pod \"7ad30631-2b97-4c5b-b220-de754ab095ba\" (UID: \"7ad30631-2b97-4c5b-b220-de754ab095ba\") "
Oct 30 22:24:01.784660 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:01.784636    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad30631-2b97-4c5b-b220-de754ab095ba-maas-api-openshift-ca" pod "7ad30631-2b97-4c5b-b220-de754ab095ba" (UID: "7ad30631-2b97-4c5b-b220-de754ab095ba"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:24:01.786321 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:01.786296    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ad30631-2b97-4c5b-b220-de754ab095ba-maas-api-xks-ca" pod "7ad30631-2b97-4c5b-b220-de754ab095ba" (UID: "7ad30631-2b97-4c5b-b220-de754ab095ba"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:24:01.786490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:01.786470    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad30631-2b97-4c5b-b220-de754ab095ba-kube-api-access-242hz" pod "7ad30631-2b97-4c5b-b220-de754ab095ba" (UID: "7ad30631-2b97-4c5b-b220-de754ab095ba"). InnerVolumeSpecName "kube-api-access-242hz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:01.885194 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:01.885150    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-242hz\" (UniqueName: \"kubernetes.io/projected/7ad30631-2b97-4c5b-b220-de754ab095ba-kube-api-access-242hz\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:01.885194 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:01.885186    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ad30631-2b97-4c5b-b220-de754ab095ba-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:01.885194 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:01.885195    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ad30631-2b97-4c5b-b220-de754ab095ba-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:02.534298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:02.534265    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"
Oct 30 22:24:02.534298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:02.534284    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z" event={"ID":"7ad30631-2b97-4c5b-b220-de754ab095ba","Type":"ContainerDied","Data":"90d7175a118587232c5b75ddf8751f1622686085f270629bbde34b6302796332"}
Oct 30 22:24:02.534527 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:02.534332    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90d7175a118587232c5b75ddf8751f1622686085f270629bbde34b6302796332"
Oct 30 22:24:10.788659 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:10.788617    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"]
Oct 30 22:24:10.789082 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:10.789069    5586 cpu_manager.go:414] "containerMap: removing container" podUID="7ad30631-2b97-4c5b-b220-de754ab095ba" containerName="revoke-keys"
Oct 30 22:24:10.789125 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:10.789084    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ad30631-2b97-4c5b-b220-de754ab095ba" containerName="revoke-keys"
Oct 30 22:24:10.789167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:10.789158    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ad30631-2b97-4c5b-b220-de754ab095ba" containerName="revoke-keys"
Oct 30 22:24:10.792620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:10.792595    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"
Oct 30 22:24:10.849147 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:10.849113    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"]
Oct 30 22:24:10.895471 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:10.895429    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:24:10.895677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:10.895539    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:24:10.895677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:10.895581    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:24:10.895677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:10.895586    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:24:10.980680 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:10.980634    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-lsvj6\" (UID: \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"
Oct 30 22:24:10.980874 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:10.980784    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-lsvj6\" (UID: \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"
Oct 30 22:24:10.980874 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:10.980822    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-922z9\" (UniqueName: \"kubernetes.io/projected/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-kube-api-access-922z9\") pod \"maas-api-revoke-sub-ec843018-lsvj6\" (UID: \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"
Oct 30 22:24:11.081621 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:11.081529    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-lsvj6\" (UID: \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"
Oct 30 22:24:11.081818 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:11.081624    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-lsvj6\" (UID: \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"
Oct 30 22:24:11.081818 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:11.081651    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-922z9\" (UniqueName: \"kubernetes.io/projected/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-kube-api-access-922z9\") pod \"maas-api-revoke-sub-ec843018-lsvj6\" (UID: \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"
Oct 30 22:24:11.082293 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:11.082265    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-lsvj6\" (UID: \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"
Oct 30 22:24:11.083997 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:11.083972    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-lsvj6\" (UID: \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"
Oct 30 22:24:11.138440 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:11.138408    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-922z9\" (UniqueName: \"kubernetes.io/projected/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-kube-api-access-922z9\") pod \"maas-api-revoke-sub-ec843018-lsvj6\" (UID: \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"
Oct 30 22:24:11.405798 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:11.405765    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"
Oct 30 22:24:11.787914 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:11.787784    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"]
Oct 30 22:24:11.790360 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:24:11.790330    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbe5ecef_5839_4740_9b27_2de61f2b0b1c.slice/crio-0d08556510740742231c186acef672c0c005ad4f74a1be31ff893fa1d8f06a1c WatchSource:0}: Error finding container 0d08556510740742231c186acef672c0c005ad4f74a1be31ff893fa1d8f06a1c: Status 404 returned error can't find the container with id 0d08556510740742231c186acef672c0c005ad4f74a1be31ff893fa1d8f06a1c
Oct 30 22:24:12.592951 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:12.592908    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6" event={"ID":"cbe5ecef-5839-4740-9b27-2de61f2b0b1c","Type":"ContainerStarted","Data":"0dbb7cc8ab0d0e6073d63023a73c787a10828da59692676f2200dd123beb12d0"}
Oct 30 22:24:12.593217 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:12.593006    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6" event={"ID":"cbe5ecef-5839-4740-9b27-2de61f2b0b1c","Type":"ContainerStarted","Data":"0d08556510740742231c186acef672c0c005ad4f74a1be31ff893fa1d8f06a1c"}
Oct 30 22:24:12.708153 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:12.708094    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6" podStartSLOduration=2.708079603 podStartE2EDuration="2.708079603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:12.706599605 +0000 UTC m=+2995.378503540" watchObservedRunningTime="2026-10-30 22:24:12.708079603 +0000 UTC m=+2995.379983537"
Oct 30 22:24:13.600081 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:13.600043    5586 generic.go:356] "Generic (PLEG): container finished" podID="cbe5ecef-5839-4740-9b27-2de61f2b0b1c" containerID="0dbb7cc8ab0d0e6073d63023a73c787a10828da59692676f2200dd123beb12d0" exitCode=0
Oct 30 22:24:13.600567 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:13.600129    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6" event={"ID":"cbe5ecef-5839-4740-9b27-2de61f2b0b1c","Type":"ContainerDied","Data":"0dbb7cc8ab0d0e6073d63023a73c787a10828da59692676f2200dd123beb12d0"}
Oct 30 22:24:14.750413 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:14.750384    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"
Oct 30 22:24:14.814714 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:14.814677    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-maas-api-xks-ca\") pod \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\" (UID: \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\") "
Oct 30 22:24:14.814905 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:14.814738    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-kube-api-access-922z9\" (UniqueName: \"kubernetes.io/projected/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-kube-api-access-922z9\") pod \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\" (UID: \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\") "
Oct 30 22:24:14.814905 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:14.814792    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-maas-api-openshift-ca\") pod \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\" (UID: \"cbe5ecef-5839-4740-9b27-2de61f2b0b1c\") "
Oct 30 22:24:14.815222 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:14.815195    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-maas-api-openshift-ca" pod "cbe5ecef-5839-4740-9b27-2de61f2b0b1c" (UID: "cbe5ecef-5839-4740-9b27-2de61f2b0b1c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:24:14.816924 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:14.816904    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-maas-api-xks-ca" pod "cbe5ecef-5839-4740-9b27-2de61f2b0b1c" (UID: "cbe5ecef-5839-4740-9b27-2de61f2b0b1c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:24:14.816998 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:14.816950    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-kube-api-access-922z9" pod "cbe5ecef-5839-4740-9b27-2de61f2b0b1c" (UID: "cbe5ecef-5839-4740-9b27-2de61f2b0b1c"). InnerVolumeSpecName "kube-api-access-922z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:14.916012 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:14.915976    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:14.916012 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:14.916008    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-922z9\" (UniqueName: \"kubernetes.io/projected/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-kube-api-access-922z9\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:14.916012 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:14.916018    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cbe5ecef-5839-4740-9b27-2de61f2b0b1c-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:24:15.613632 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:15.613602    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"
Oct 30 22:24:15.613820 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:15.613641    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6" event={"ID":"cbe5ecef-5839-4740-9b27-2de61f2b0b1c","Type":"ContainerDied","Data":"0d08556510740742231c186acef672c0c005ad4f74a1be31ff893fa1d8f06a1c"}
Oct 30 22:24:15.613820 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:15.613690    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d08556510740742231c186acef672c0c005ad4f74a1be31ff893fa1d8f06a1c"
Oct 30 22:24:18.072954 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:18.072924    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:24:18.077957 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:18.077929    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:24:18.089700 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:18.089665    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:24:18.094508 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:18.094484    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:24:18.360747 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:24:18.360659    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:24:33.058362 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:24:33.058325    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:48.360344 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:24:48.360307    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:25:06.156420 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.156387    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"]
Oct 30 22:25:06.156935 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.156858    5586 cpu_manager.go:414] "containerMap: removing container" podUID="cbe5ecef-5839-4740-9b27-2de61f2b0b1c" containerName="revoke-keys"
Oct 30 22:25:06.156935 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.156875    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cbe5ecef-5839-4740-9b27-2de61f2b0b1c" containerName="revoke-keys"
Oct 30 22:25:06.157026 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.156964    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="cbe5ecef-5839-4740-9b27-2de61f2b0b1c" containerName="revoke-keys"
Oct 30 22:25:06.160294 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.160269    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"
Oct 30 22:25:06.214980 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.214944    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:25:06.218332 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.218299    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"]
Oct 30 22:25:06.264066 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.264036    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:25:06.264066 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.264048    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:25:06.264291 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.264101    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:25:06.302503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.302467    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-kvftd\" (UID: \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"
Oct 30 22:25:06.302688 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.302519    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-kvftd\" (UID: \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"
Oct 30 22:25:06.302688 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.302590    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrpn4\" (UniqueName: \"kubernetes.io/projected/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-kube-api-access-lrpn4\") pod \"maas-api-revoke-sub-046e1343-kvftd\" (UID: \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"
Oct 30 22:25:06.403372 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.403335    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-kvftd\" (UID: \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"
Oct 30 22:25:06.403566 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.403389    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-kvftd\" (UID: \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"
Oct 30 22:25:06.403566 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.403426    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrpn4\" (UniqueName: \"kubernetes.io/projected/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-kube-api-access-lrpn4\") pod \"maas-api-revoke-sub-046e1343-kvftd\" (UID: \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"
Oct 30 22:25:06.404116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.404093    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-kvftd\" (UID: \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"
Oct 30 22:25:06.405705 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.405686    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-kvftd\" (UID: \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"
Oct 30 22:25:06.461420 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.461339    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrpn4\" (UniqueName: \"kubernetes.io/projected/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-kube-api-access-lrpn4\") pod \"maas-api-revoke-sub-046e1343-kvftd\" (UID: \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"
Oct 30 22:25:06.473757 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.473721    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"
Oct 30 22:25:06.847125 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.847095    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"]
Oct 30 22:25:06.849602 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:25:06.849561    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c1b51ea_d80f_4aaa_80e2_564e6aa7eff4.slice/crio-5a64c076dd5211418dbfa0369e15c434ac19c3660813caa87f57f6b93dc81eba WatchSource:0}: Error finding container 5a64c076dd5211418dbfa0369e15c434ac19c3660813caa87f57f6b93dc81eba: Status 404 returned error can't find the container with id 5a64c076dd5211418dbfa0369e15c434ac19c3660813caa87f57f6b93dc81eba
Oct 30 22:25:06.942086 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:06.942045    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd" event={"ID":"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4","Type":"ContainerStarted","Data":"5a64c076dd5211418dbfa0369e15c434ac19c3660813caa87f57f6b93dc81eba"}
Oct 30 22:25:07.951589 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:07.951546    5586 generic.go:356] "Generic (PLEG): container finished" podID="2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4" containerID="ff5301b389db65116b8e8b372b561270c3fd139308db33a382cb40f84a868a7d" exitCode=0
Oct 30 22:25:07.952078 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:07.951813    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd" event={"ID":"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4","Type":"ContainerDied","Data":"ff5301b389db65116b8e8b372b561270c3fd139308db33a382cb40f84a868a7d"}
Oct 30 22:25:09.092155 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:09.092124    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"
Oct 30 22:25:09.230950 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:09.230858    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-maas-api-xks-ca\") pod \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\" (UID: \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\") "
Oct 30 22:25:09.231107 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:09.231032    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-kube-api-access-lrpn4\" (UniqueName: \"kubernetes.io/projected/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-kube-api-access-lrpn4\") pod \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\" (UID: \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\") "
Oct 30 22:25:09.231107 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:09.231094    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-maas-api-openshift-ca\") pod \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\" (UID: \"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4\") "
Oct 30 22:25:09.231547 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:09.231508    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-maas-api-openshift-ca" pod "2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4" (UID: "2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:25:09.233115 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:09.233079    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-maas-api-xks-ca" pod "2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4" (UID: "2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:25:09.233249 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:09.233212    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-kube-api-access-lrpn4" pod "2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4" (UID: "2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4"). InnerVolumeSpecName "kube-api-access-lrpn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:09.332206 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:09.332157    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrpn4\" (UniqueName: \"kubernetes.io/projected/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-kube-api-access-lrpn4\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:25:09.332206 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:09.332194    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:25:09.332206 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:09.332205    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:25:09.964071 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:09.964037    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"
Oct 30 22:25:09.964278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:09.964045    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd" event={"ID":"2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4","Type":"ContainerDied","Data":"5a64c076dd5211418dbfa0369e15c434ac19c3660813caa87f57f6b93dc81eba"}
Oct 30 22:25:09.964278 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:09.964162    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a64c076dd5211418dbfa0369e15c434ac19c3660813caa87f57f6b93dc81eba"
Oct 30 22:25:16.921109 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:16.921067    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"]
Oct 30 22:25:16.921795 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:16.921773    5586 cpu_manager.go:414] "containerMap: removing container" podUID="2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4" containerName="revoke-keys"
Oct 30 22:25:16.921839 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:16.921799    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4" containerName="revoke-keys"
Oct 30 22:25:16.921945 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:16.921934    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4" containerName="revoke-keys"
Oct 30 22:25:16.925921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:16.925899    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"
Oct 30 22:25:16.981855 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:16.981819    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"]
Oct 30 22:25:17.011062 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.011015    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52b4t\" (UniqueName: \"kubernetes.io/projected/20a60a01-46ac-45b4-8530-24621cbcea7a-kube-api-access-52b4t\") pod \"maas-api-revoke-sub-d3fba056-6fz97\" (UID: \"20a60a01-46ac-45b4-8530-24621cbcea7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"
Oct 30 22:25:17.011062 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.011064    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/20a60a01-46ac-45b4-8530-24621cbcea7a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-6fz97\" (UID: \"20a60a01-46ac-45b4-8530-24621cbcea7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"
Oct 30 22:25:17.011306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.011173    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/20a60a01-46ac-45b4-8530-24621cbcea7a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-6fz97\" (UID: \"20a60a01-46ac-45b4-8530-24621cbcea7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"
Oct 30 22:25:17.027702 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.027669    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:25:17.027855 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.027724    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:25:17.027855 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.027733    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:25:17.027855 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.027778    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:25:17.112229 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.112187    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/20a60a01-46ac-45b4-8530-24621cbcea7a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-6fz97\" (UID: \"20a60a01-46ac-45b4-8530-24621cbcea7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"
Oct 30 22:25:17.112446 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.112315    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52b4t\" (UniqueName: \"kubernetes.io/projected/20a60a01-46ac-45b4-8530-24621cbcea7a-kube-api-access-52b4t\") pod \"maas-api-revoke-sub-d3fba056-6fz97\" (UID: \"20a60a01-46ac-45b4-8530-24621cbcea7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"
Oct 30 22:25:17.112446 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.112350    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/20a60a01-46ac-45b4-8530-24621cbcea7a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-6fz97\" (UID: \"20a60a01-46ac-45b4-8530-24621cbcea7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"
Oct 30 22:25:17.112851 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.112827    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/20a60a01-46ac-45b4-8530-24621cbcea7a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-6fz97\" (UID: \"20a60a01-46ac-45b4-8530-24621cbcea7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"
Oct 30 22:25:17.114816 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.114793    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/20a60a01-46ac-45b4-8530-24621cbcea7a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-6fz97\" (UID: \"20a60a01-46ac-45b4-8530-24621cbcea7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"
Oct 30 22:25:17.170597 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.170568    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52b4t\" (UniqueName: \"kubernetes.io/projected/20a60a01-46ac-45b4-8530-24621cbcea7a-kube-api-access-52b4t\") pod \"maas-api-revoke-sub-d3fba056-6fz97\" (UID: \"20a60a01-46ac-45b4-8530-24621cbcea7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"
Oct 30 22:25:17.239476 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.239374    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"
Oct 30 22:25:17.632349 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:17.632319    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"]
Oct 30 22:25:17.634475 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:25:17.634437    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a60a01_46ac_45b4_8530_24621cbcea7a.slice/crio-6a3fe7ebebceb917a2c0d33a905d0f402d895e6224dc7ff35c453da8ffa3dfe2 WatchSource:0}: Error finding container 6a3fe7ebebceb917a2c0d33a905d0f402d895e6224dc7ff35c453da8ffa3dfe2: Status 404 returned error can't find the container with id 6a3fe7ebebceb917a2c0d33a905d0f402d895e6224dc7ff35c453da8ffa3dfe2
Oct 30 22:25:18.010719 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:18.010682    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97" event={"ID":"20a60a01-46ac-45b4-8530-24621cbcea7a","Type":"ContainerStarted","Data":"910e2a63b3e85b2b6767358fe936408a4aed5a11556f009c5c8fdf41ef237426"}
Oct 30 22:25:18.011132 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:18.010733    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97" event={"ID":"20a60a01-46ac-45b4-8530-24621cbcea7a","Type":"ContainerStarted","Data":"6a3fe7ebebceb917a2c0d33a905d0f402d895e6224dc7ff35c453da8ffa3dfe2"}
Oct 30 22:25:18.125436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:18.125374    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97" podStartSLOduration=2.125356591 podStartE2EDuration="2.125356591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:25:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:18.125317112 +0000 UTC m=+3060.797221048" watchObservedRunningTime="2026-10-30 22:25:18.125356591 +0000 UTC m=+3060.797260529"
Oct 30 22:25:18.361419 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:25:18.361333    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:25:19.018859 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:19.018821    5586 generic.go:356] "Generic (PLEG): container finished" podID="20a60a01-46ac-45b4-8530-24621cbcea7a" containerID="910e2a63b3e85b2b6767358fe936408a4aed5a11556f009c5c8fdf41ef237426" exitCode=0
Oct 30 22:25:19.019281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:19.018875    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97" event={"ID":"20a60a01-46ac-45b4-8530-24621cbcea7a","Type":"ContainerDied","Data":"910e2a63b3e85b2b6767358fe936408a4aed5a11556f009c5c8fdf41ef237426"}
Oct 30 22:25:20.248871 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:20.248837    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"
Oct 30 22:25:20.341318 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:20.340527    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20a60a01-46ac-45b4-8530-24621cbcea7a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/20a60a01-46ac-45b4-8530-24621cbcea7a-maas-api-openshift-ca\") pod \"20a60a01-46ac-45b4-8530-24621cbcea7a\" (UID: \"20a60a01-46ac-45b4-8530-24621cbcea7a\") "
Oct 30 22:25:20.341318 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:20.340623    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20a60a01-46ac-45b4-8530-24621cbcea7a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/20a60a01-46ac-45b4-8530-24621cbcea7a-maas-api-xks-ca\") pod \"20a60a01-46ac-45b4-8530-24621cbcea7a\" (UID: \"20a60a01-46ac-45b4-8530-24621cbcea7a\") "
Oct 30 22:25:20.341318 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:20.340702    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20a60a01-46ac-45b4-8530-24621cbcea7a-kube-api-access-52b4t\" (UniqueName: \"kubernetes.io/projected/20a60a01-46ac-45b4-8530-24621cbcea7a-kube-api-access-52b4t\") pod \"20a60a01-46ac-45b4-8530-24621cbcea7a\" (UID: \"20a60a01-46ac-45b4-8530-24621cbcea7a\") "
Oct 30 22:25:20.343794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:20.343755    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20a60a01-46ac-45b4-8530-24621cbcea7a-maas-api-openshift-ca" pod "20a60a01-46ac-45b4-8530-24621cbcea7a" (UID: "20a60a01-46ac-45b4-8530-24621cbcea7a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:25:20.353494 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:20.353431    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a60a01-46ac-45b4-8530-24621cbcea7a-kube-api-access-52b4t" pod "20a60a01-46ac-45b4-8530-24621cbcea7a" (UID: "20a60a01-46ac-45b4-8530-24621cbcea7a"). InnerVolumeSpecName "kube-api-access-52b4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:20.361490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:20.361431    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20a60a01-46ac-45b4-8530-24621cbcea7a-maas-api-xks-ca" pod "20a60a01-46ac-45b4-8530-24621cbcea7a" (UID: "20a60a01-46ac-45b4-8530-24621cbcea7a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:25:20.486259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:20.482493    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/20a60a01-46ac-45b4-8530-24621cbcea7a-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:25:20.486259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:20.482542    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/20a60a01-46ac-45b4-8530-24621cbcea7a-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:25:20.486259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:20.482558    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52b4t\" (UniqueName: \"kubernetes.io/projected/20a60a01-46ac-45b4-8530-24621cbcea7a-kube-api-access-52b4t\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:25:21.040259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:21.022143    5586 scope.go:122] "RemoveContainer" containerID="283191a7da3443481d59b9bb66f0f323df2c46829755beb219d69505d89e3fd9"
Oct 30 22:25:21.127696 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:21.120002    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97" event={"ID":"20a60a01-46ac-45b4-8530-24621cbcea7a","Type":"ContainerDied","Data":"6a3fe7ebebceb917a2c0d33a905d0f402d895e6224dc7ff35c453da8ffa3dfe2"}
Oct 30 22:25:21.127696 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:21.120068    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a3fe7ebebceb917a2c0d33a905d0f402d895e6224dc7ff35c453da8ffa3dfe2"
Oct 30 22:25:21.127696 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:21.120146    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"
Oct 30 22:25:26.009936 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:26.009897    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:33.320546 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:33.320447    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:48.361848 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:25:48.361805    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:25:51.751226 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.751185    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"]
Oct 30 22:25:51.751676 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.751646    5586 cpu_manager.go:414] "containerMap: removing container" podUID="20a60a01-46ac-45b4-8530-24621cbcea7a" containerName="revoke-keys"
Oct 30 22:25:51.751676 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.751658    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20a60a01-46ac-45b4-8530-24621cbcea7a" containerName="revoke-keys"
Oct 30 22:25:51.751760 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.751730    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="20a60a01-46ac-45b4-8530-24621cbcea7a" containerName="revoke-keys"
Oct 30 22:25:51.754882 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.754864    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"
Oct 30 22:25:51.847474 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.847434    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ed2517b8-7ae0-43b0-9724-15a73df80e70-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-748mh\" (UID: \"ed2517b8-7ae0-43b0-9724-15a73df80e70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"
Oct 30 22:25:51.847686 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.847523    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ed2517b8-7ae0-43b0-9724-15a73df80e70-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-748mh\" (UID: \"ed2517b8-7ae0-43b0-9724-15a73df80e70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"
Oct 30 22:25:51.847686 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.847557    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph7n4\" (UniqueName: \"kubernetes.io/projected/ed2517b8-7ae0-43b0-9724-15a73df80e70-kube-api-access-ph7n4\") pod \"maas-api-revoke-sub-ae594979-748mh\" (UID: \"ed2517b8-7ae0-43b0-9724-15a73df80e70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"
Oct 30 22:25:51.857889 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.857855    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:25:51.857889 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.857859    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:25:51.858117 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.857913    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:25:51.858117 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.857936    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:25:51.860389 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.860363    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"]
Oct 30 22:25:51.948168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.948129    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ed2517b8-7ae0-43b0-9724-15a73df80e70-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-748mh\" (UID: \"ed2517b8-7ae0-43b0-9724-15a73df80e70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"
Oct 30 22:25:51.948168 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.948176    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph7n4\" (UniqueName: \"kubernetes.io/projected/ed2517b8-7ae0-43b0-9724-15a73df80e70-kube-api-access-ph7n4\") pod \"maas-api-revoke-sub-ae594979-748mh\" (UID: \"ed2517b8-7ae0-43b0-9724-15a73df80e70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"
Oct 30 22:25:51.948493 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.948314    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ed2517b8-7ae0-43b0-9724-15a73df80e70-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-748mh\" (UID: \"ed2517b8-7ae0-43b0-9724-15a73df80e70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"
Oct 30 22:25:51.948887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.948859    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ed2517b8-7ae0-43b0-9724-15a73df80e70-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-748mh\" (UID: \"ed2517b8-7ae0-43b0-9724-15a73df80e70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"
Oct 30 22:25:51.950680 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:51.950652    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ed2517b8-7ae0-43b0-9724-15a73df80e70-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-748mh\" (UID: \"ed2517b8-7ae0-43b0-9724-15a73df80e70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"
Oct 30 22:25:52.006203 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:52.006122    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph7n4\" (UniqueName: \"kubernetes.io/projected/ed2517b8-7ae0-43b0-9724-15a73df80e70-kube-api-access-ph7n4\") pod \"maas-api-revoke-sub-ae594979-748mh\" (UID: \"ed2517b8-7ae0-43b0-9724-15a73df80e70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"
Oct 30 22:25:52.068210 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:52.068174    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"
Oct 30 22:25:52.095874 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:52.095840    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:52.262022 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:52.261925    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:52.450673 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:52.450641    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"]
Oct 30 22:25:52.452265 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:25:52.452213    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded2517b8_7ae0_43b0_9724_15a73df80e70.slice/crio-6be6b008bd52440b1565a35d0c76d5a53f97ddf8e249f1a2453bbaa3c8e65ea0 WatchSource:0}: Error finding container 6be6b008bd52440b1565a35d0c76d5a53f97ddf8e249f1a2453bbaa3c8e65ea0: Status 404 returned error can't find the container with id 6be6b008bd52440b1565a35d0c76d5a53f97ddf8e249f1a2453bbaa3c8e65ea0
Oct 30 22:25:53.404358 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:53.404324    5586 generic.go:356] "Generic (PLEG): container finished" podID="ed2517b8-7ae0-43b0-9724-15a73df80e70" containerID="2cb77808e69752a486fbb8e77cf373e297f199f4c32954f96469710dc31ded57" exitCode=0
Oct 30 22:25:53.404780 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:53.404410    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh" event={"ID":"ed2517b8-7ae0-43b0-9724-15a73df80e70","Type":"ContainerDied","Data":"2cb77808e69752a486fbb8e77cf373e297f199f4c32954f96469710dc31ded57"}
Oct 30 22:25:53.404780 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:53.404464    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh" event={"ID":"ed2517b8-7ae0-43b0-9724-15a73df80e70","Type":"ContainerStarted","Data":"6be6b008bd52440b1565a35d0c76d5a53f97ddf8e249f1a2453bbaa3c8e65ea0"}
Oct 30 22:25:54.547220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:54.547194    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"
Oct 30 22:25:54.571181 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:54.571145    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed2517b8-7ae0-43b0-9724-15a73df80e70-kube-api-access-ph7n4\" (UniqueName: \"kubernetes.io/projected/ed2517b8-7ae0-43b0-9724-15a73df80e70-kube-api-access-ph7n4\") pod \"ed2517b8-7ae0-43b0-9724-15a73df80e70\" (UID: \"ed2517b8-7ae0-43b0-9724-15a73df80e70\") "
Oct 30 22:25:54.571372 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:54.571256    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed2517b8-7ae0-43b0-9724-15a73df80e70-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ed2517b8-7ae0-43b0-9724-15a73df80e70-maas-api-openshift-ca\") pod \"ed2517b8-7ae0-43b0-9724-15a73df80e70\" (UID: \"ed2517b8-7ae0-43b0-9724-15a73df80e70\") "
Oct 30 22:25:54.571372 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:54.571320    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ed2517b8-7ae0-43b0-9724-15a73df80e70-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ed2517b8-7ae0-43b0-9724-15a73df80e70-maas-api-xks-ca\") pod \"ed2517b8-7ae0-43b0-9724-15a73df80e70\" (UID: \"ed2517b8-7ae0-43b0-9724-15a73df80e70\") "
Oct 30 22:25:54.571694 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:54.571666    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed2517b8-7ae0-43b0-9724-15a73df80e70-maas-api-openshift-ca" pod "ed2517b8-7ae0-43b0-9724-15a73df80e70" (UID: "ed2517b8-7ae0-43b0-9724-15a73df80e70"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:25:54.573297 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:54.573274    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed2517b8-7ae0-43b0-9724-15a73df80e70-maas-api-xks-ca" pod "ed2517b8-7ae0-43b0-9724-15a73df80e70" (UID: "ed2517b8-7ae0-43b0-9724-15a73df80e70"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:25:54.573375 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:54.573320    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed2517b8-7ae0-43b0-9724-15a73df80e70-kube-api-access-ph7n4" pod "ed2517b8-7ae0-43b0-9724-15a73df80e70" (UID: "ed2517b8-7ae0-43b0-9724-15a73df80e70"). InnerVolumeSpecName "kube-api-access-ph7n4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:54.672121 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:54.672038    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ed2517b8-7ae0-43b0-9724-15a73df80e70-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:25:54.672121 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:54.672068    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ph7n4\" (UniqueName: \"kubernetes.io/projected/ed2517b8-7ae0-43b0-9724-15a73df80e70-kube-api-access-ph7n4\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:25:54.672121 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:54.672080    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ed2517b8-7ae0-43b0-9724-15a73df80e70-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:25:55.416756 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:55.416722    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"
Oct 30 22:25:55.416954 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:55.416751    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh" event={"ID":"ed2517b8-7ae0-43b0-9724-15a73df80e70","Type":"ContainerDied","Data":"6be6b008bd52440b1565a35d0c76d5a53f97ddf8e249f1a2453bbaa3c8e65ea0"}
Oct 30 22:25:55.416954 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:25:55.416834    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6be6b008bd52440b1565a35d0c76d5a53f97ddf8e249f1a2453bbaa3c8e65ea0"
Oct 30 22:26:00.856179 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:00.856145    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:12.115270 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:12.114349    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-46ldm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.162:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:26:12.115270 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:12.114437    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" podUID="b1dbe21e-18b9-4af9-b2c2-e36c6408543b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.162:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:26:12.115270 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:12.114836    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-wwv4p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:26:12.115270 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:12.114880    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" podUID="7e2fe270-2a33-4648-96a6-b0486a69350d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:26:12.261628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:12.261537    5586 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-55hwj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:26:12.261965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:12.261933    5586 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" podUID="c88a6fd6-2cac-4ad4-b523-905ed7d62bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:26:18.361216 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:26:18.361171    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:26:20.076961 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:20.076917    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"]
Oct 30 22:26:20.124341 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:20.124308    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-866238f7-2fghq"]
Oct 30 22:26:21.445942 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:21.445907    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1687120-f405-4ac3-9a97-2d9490acd9d5" path="/var/lib/kubelet/pods/d1687120-f405-4ac3-9a97-2d9490acd9d5/volumes"
Oct 30 22:26:29.925303 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:29.925265    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"]
Oct 30 22:26:29.925732 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:29.925698    5586 cpu_manager.go:414] "containerMap: removing container" podUID="ed2517b8-7ae0-43b0-9724-15a73df80e70" containerName="revoke-keys"
Oct 30 22:26:29.925732 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:29.925709    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed2517b8-7ae0-43b0-9724-15a73df80e70" containerName="revoke-keys"
Oct 30 22:26:29.925824 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:29.925785    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed2517b8-7ae0-43b0-9724-15a73df80e70" containerName="revoke-keys"
Oct 30 22:26:29.929420 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:29.929399    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"
Oct 30 22:26:29.956514 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:29.956483    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0db15b9e-2e58-432a-990b-fbe7946d9f33-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-vhsl4\" (UID: \"0db15b9e-2e58-432a-990b-fbe7946d9f33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"
Oct 30 22:26:29.956678 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:29.956530    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0db15b9e-2e58-432a-990b-fbe7946d9f33-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-vhsl4\" (UID: \"0db15b9e-2e58-432a-990b-fbe7946d9f33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"
Oct 30 22:26:29.956678 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:29.956636    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7mtt\" (UniqueName: \"kubernetes.io/projected/0db15b9e-2e58-432a-990b-fbe7946d9f33-kube-api-access-c7mtt\") pod \"maas-api-revoke-sub-a4aef7d7-vhsl4\" (UID: \"0db15b9e-2e58-432a-990b-fbe7946d9f33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"
Oct 30 22:26:29.992592 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:29.992560    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"]
Oct 30 22:26:30.036984 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:30.036948    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:26:30.037188 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:30.037169    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:26:30.037279 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:30.037208    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:26:30.039725 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:30.039705    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:26:30.057807 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:30.057772    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0db15b9e-2e58-432a-990b-fbe7946d9f33-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-vhsl4\" (UID: \"0db15b9e-2e58-432a-990b-fbe7946d9f33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"
Oct 30 22:26:30.057974 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:30.057820    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0db15b9e-2e58-432a-990b-fbe7946d9f33-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-vhsl4\" (UID: \"0db15b9e-2e58-432a-990b-fbe7946d9f33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"
Oct 30 22:26:30.057974 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:30.057851    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7mtt\" (UniqueName: \"kubernetes.io/projected/0db15b9e-2e58-432a-990b-fbe7946d9f33-kube-api-access-c7mtt\") pod \"maas-api-revoke-sub-a4aef7d7-vhsl4\" (UID: \"0db15b9e-2e58-432a-990b-fbe7946d9f33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"
Oct 30 22:26:30.058529 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:30.058503    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0db15b9e-2e58-432a-990b-fbe7946d9f33-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-vhsl4\" (UID: \"0db15b9e-2e58-432a-990b-fbe7946d9f33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"
Oct 30 22:26:30.060312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:30.060292    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0db15b9e-2e58-432a-990b-fbe7946d9f33-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-vhsl4\" (UID: \"0db15b9e-2e58-432a-990b-fbe7946d9f33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"
Oct 30 22:26:30.115357 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:30.115330    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7mtt\" (UniqueName: \"kubernetes.io/projected/0db15b9e-2e58-432a-990b-fbe7946d9f33-kube-api-access-c7mtt\") pod \"maas-api-revoke-sub-a4aef7d7-vhsl4\" (UID: \"0db15b9e-2e58-432a-990b-fbe7946d9f33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"
Oct 30 22:26:30.242813 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:30.242713    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"
Oct 30 22:26:30.618897 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:30.618813    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"]
Oct 30 22:26:30.622029 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:26:30.621987    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db15b9e_2e58_432a_990b_fbe7946d9f33.slice/crio-a20fc9eaf92f37b95a0d1863b9da303529cac530489f474db39e6a207bedc773 WatchSource:0}: Error finding container a20fc9eaf92f37b95a0d1863b9da303529cac530489f474db39e6a207bedc773: Status 404 returned error can't find the container with id a20fc9eaf92f37b95a0d1863b9da303529cac530489f474db39e6a207bedc773
Oct 30 22:26:30.659605 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:30.659572    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4" event={"ID":"0db15b9e-2e58-432a-990b-fbe7946d9f33","Type":"ContainerStarted","Data":"a20fc9eaf92f37b95a0d1863b9da303529cac530489f474db39e6a207bedc773"}
Oct 30 22:26:31.667169 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:31.667136    5586 generic.go:356] "Generic (PLEG): container finished" podID="0db15b9e-2e58-432a-990b-fbe7946d9f33" containerID="e55b776ae42576ed29581a8b863935c129d4d8f3869d51c99408231c575c598e" exitCode=0
Oct 30 22:26:31.667634 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:31.667224    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4" event={"ID":"0db15b9e-2e58-432a-990b-fbe7946d9f33","Type":"ContainerDied","Data":"e55b776ae42576ed29581a8b863935c129d4d8f3869d51c99408231c575c598e"}
Oct 30 22:26:32.814335 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:32.814308    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"
Oct 30 22:26:32.985297 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:32.985176    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0db15b9e-2e58-432a-990b-fbe7946d9f33-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0db15b9e-2e58-432a-990b-fbe7946d9f33-maas-api-xks-ca\") pod \"0db15b9e-2e58-432a-990b-fbe7946d9f33\" (UID: \"0db15b9e-2e58-432a-990b-fbe7946d9f33\") "
Oct 30 22:26:32.985468 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:32.985307    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0db15b9e-2e58-432a-990b-fbe7946d9f33-kube-api-access-c7mtt\" (UniqueName: \"kubernetes.io/projected/0db15b9e-2e58-432a-990b-fbe7946d9f33-kube-api-access-c7mtt\") pod \"0db15b9e-2e58-432a-990b-fbe7946d9f33\" (UID: \"0db15b9e-2e58-432a-990b-fbe7946d9f33\") "
Oct 30 22:26:32.985468 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:32.985369    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0db15b9e-2e58-432a-990b-fbe7946d9f33-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0db15b9e-2e58-432a-990b-fbe7946d9f33-maas-api-openshift-ca\") pod \"0db15b9e-2e58-432a-990b-fbe7946d9f33\" (UID: \"0db15b9e-2e58-432a-990b-fbe7946d9f33\") "
Oct 30 22:26:32.985767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:32.985731    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0db15b9e-2e58-432a-990b-fbe7946d9f33-maas-api-openshift-ca" pod "0db15b9e-2e58-432a-990b-fbe7946d9f33" (UID: "0db15b9e-2e58-432a-990b-fbe7946d9f33"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:32.987460 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:32.987435    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0db15b9e-2e58-432a-990b-fbe7946d9f33-maas-api-xks-ca" pod "0db15b9e-2e58-432a-990b-fbe7946d9f33" (UID: "0db15b9e-2e58-432a-990b-fbe7946d9f33"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:26:32.987578 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:32.987477    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0db15b9e-2e58-432a-990b-fbe7946d9f33-kube-api-access-c7mtt" pod "0db15b9e-2e58-432a-990b-fbe7946d9f33" (UID: "0db15b9e-2e58-432a-990b-fbe7946d9f33"). InnerVolumeSpecName "kube-api-access-c7mtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:33.086450 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:33.086407    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0db15b9e-2e58-432a-990b-fbe7946d9f33-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:33.086450 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:33.086440    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0db15b9e-2e58-432a-990b-fbe7946d9f33-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:33.086450 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:33.086450    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7mtt\" (UniqueName: \"kubernetes.io/projected/0db15b9e-2e58-432a-990b-fbe7946d9f33-kube-api-access-c7mtt\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:33.679685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:33.679652    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"
Oct 30 22:26:33.679872 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:33.679651    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4" event={"ID":"0db15b9e-2e58-432a-990b-fbe7946d9f33","Type":"ContainerDied","Data":"a20fc9eaf92f37b95a0d1863b9da303529cac530489f474db39e6a207bedc773"}
Oct 30 22:26:33.679872 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:33.679778    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a20fc9eaf92f37b95a0d1863b9da303529cac530489f474db39e6a207bedc773"
Oct 30 22:26:35.242497 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:26:35.242452    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:48.361026 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:26:48.360987    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:27:07.386478 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.386249    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"]
Oct 30 22:27:07.387197 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.386881    5586 cpu_manager.go:414] "containerMap: removing container" podUID="0db15b9e-2e58-432a-990b-fbe7946d9f33" containerName="revoke-keys"
Oct 30 22:27:07.387197 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.386903    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0db15b9e-2e58-432a-990b-fbe7946d9f33" containerName="revoke-keys"
Oct 30 22:27:07.387197 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.387017    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="0db15b9e-2e58-432a-990b-fbe7946d9f33" containerName="revoke-keys"
Oct 30 22:27:07.390211 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.390194    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"
Oct 30 22:27:07.448957 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.448919    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"]
Oct 30 22:27:07.470419 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.470375    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tng4r\" (UniqueName: \"kubernetes.io/projected/68613646-0636-483c-90cb-e73f7a6bfc8d-kube-api-access-tng4r\") pod \"maas-api-revoke-sub-79805ca9-n6hfm\" (UID: \"68613646-0636-483c-90cb-e73f7a6bfc8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"
Oct 30 22:27:07.470595 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.470480    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68613646-0636-483c-90cb-e73f7a6bfc8d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-n6hfm\" (UID: \"68613646-0636-483c-90cb-e73f7a6bfc8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"
Oct 30 22:27:07.470641 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.470596    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68613646-0636-483c-90cb-e73f7a6bfc8d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-n6hfm\" (UID: \"68613646-0636-483c-90cb-e73f7a6bfc8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"
Oct 30 22:27:07.492478 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.492438    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:27:07.492478 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.492459    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:27:07.492478 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.492458    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:27:07.492730 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.492496    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:27:07.571805 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.571766    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68613646-0636-483c-90cb-e73f7a6bfc8d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-n6hfm\" (UID: \"68613646-0636-483c-90cb-e73f7a6bfc8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"
Oct 30 22:27:07.572032 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.571843    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tng4r\" (UniqueName: \"kubernetes.io/projected/68613646-0636-483c-90cb-e73f7a6bfc8d-kube-api-access-tng4r\") pod \"maas-api-revoke-sub-79805ca9-n6hfm\" (UID: \"68613646-0636-483c-90cb-e73f7a6bfc8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"
Oct 30 22:27:07.572032 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.571885    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68613646-0636-483c-90cb-e73f7a6bfc8d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-n6hfm\" (UID: \"68613646-0636-483c-90cb-e73f7a6bfc8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"
Oct 30 22:27:07.572469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.572442    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68613646-0636-483c-90cb-e73f7a6bfc8d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-n6hfm\" (UID: \"68613646-0636-483c-90cb-e73f7a6bfc8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"
Oct 30 22:27:07.574340 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.574317    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68613646-0636-483c-90cb-e73f7a6bfc8d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-n6hfm\" (UID: \"68613646-0636-483c-90cb-e73f7a6bfc8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"
Oct 30 22:27:07.628939 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.628907    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tng4r\" (UniqueName: \"kubernetes.io/projected/68613646-0636-483c-90cb-e73f7a6bfc8d-kube-api-access-tng4r\") pod \"maas-api-revoke-sub-79805ca9-n6hfm\" (UID: \"68613646-0636-483c-90cb-e73f7a6bfc8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"
Oct 30 22:27:07.704368 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:07.704327    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"
Oct 30 22:27:08.082473 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:08.082446    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"]
Oct 30 22:27:08.084958 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:27:08.084929    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68613646_0636_483c_90cb_e73f7a6bfc8d.slice/crio-0f39a9cf3821b8324b98112f322d9a2d9392212cd23b181f67bc5ccbc52a0dc8 WatchSource:0}: Error finding container 0f39a9cf3821b8324b98112f322d9a2d9392212cd23b181f67bc5ccbc52a0dc8: Status 404 returned error can't find the container with id 0f39a9cf3821b8324b98112f322d9a2d9392212cd23b181f67bc5ccbc52a0dc8
Oct 30 22:27:08.922912 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:08.922870    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm" event={"ID":"68613646-0636-483c-90cb-e73f7a6bfc8d","Type":"ContainerStarted","Data":"ca4a9d7fd7698b8b6d00b524b383196af1d822066c1810e3ae7d266b51bbf436"}
Oct 30 22:27:08.923324 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:08.922948    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm" event={"ID":"68613646-0636-483c-90cb-e73f7a6bfc8d","Type":"ContainerStarted","Data":"0f39a9cf3821b8324b98112f322d9a2d9392212cd23b181f67bc5ccbc52a0dc8"}
Oct 30 22:27:09.037956 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:09.037900    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm" podStartSLOduration=2.037883385 podStartE2EDuration="2.037883385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:27:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:27:09.036155755 +0000 UTC m=+3171.708059685" watchObservedRunningTime="2026-10-30 22:27:09.037883385 +0000 UTC m=+3171.709787321"
Oct 30 22:27:09.929993 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:09.929958    5586 generic.go:356] "Generic (PLEG): container finished" podID="68613646-0636-483c-90cb-e73f7a6bfc8d" containerID="ca4a9d7fd7698b8b6d00b524b383196af1d822066c1810e3ae7d266b51bbf436" exitCode=0
Oct 30 22:27:09.930418 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:09.930050    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm" event={"ID":"68613646-0636-483c-90cb-e73f7a6bfc8d","Type":"ContainerDied","Data":"ca4a9d7fd7698b8b6d00b524b383196af1d822066c1810e3ae7d266b51bbf436"}
Oct 30 22:27:11.085034 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:11.085003    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"
Oct 30 22:27:11.105282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:11.105221    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/68613646-0636-483c-90cb-e73f7a6bfc8d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68613646-0636-483c-90cb-e73f7a6bfc8d-maas-api-xks-ca\") pod \"68613646-0636-483c-90cb-e73f7a6bfc8d\" (UID: \"68613646-0636-483c-90cb-e73f7a6bfc8d\") "
Oct 30 22:27:11.105449 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:11.105300    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68613646-0636-483c-90cb-e73f7a6bfc8d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68613646-0636-483c-90cb-e73f7a6bfc8d-maas-api-openshift-ca\") pod \"68613646-0636-483c-90cb-e73f7a6bfc8d\" (UID: \"68613646-0636-483c-90cb-e73f7a6bfc8d\") "
Oct 30 22:27:11.105449 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:11.105373    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68613646-0636-483c-90cb-e73f7a6bfc8d-kube-api-access-tng4r\" (UniqueName: \"kubernetes.io/projected/68613646-0636-483c-90cb-e73f7a6bfc8d-kube-api-access-tng4r\") pod \"68613646-0636-483c-90cb-e73f7a6bfc8d\" (UID: \"68613646-0636-483c-90cb-e73f7a6bfc8d\") "
Oct 30 22:27:11.105684 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:11.105655    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68613646-0636-483c-90cb-e73f7a6bfc8d-maas-api-openshift-ca" pod "68613646-0636-483c-90cb-e73f7a6bfc8d" (UID: "68613646-0636-483c-90cb-e73f7a6bfc8d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:27:11.105825 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:11.105781    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/68613646-0636-483c-90cb-e73f7a6bfc8d-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:11.107410 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:11.107385    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68613646-0636-483c-90cb-e73f7a6bfc8d-maas-api-xks-ca" pod "68613646-0636-483c-90cb-e73f7a6bfc8d" (UID: "68613646-0636-483c-90cb-e73f7a6bfc8d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:27:11.107505 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:11.107486    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68613646-0636-483c-90cb-e73f7a6bfc8d-kube-api-access-tng4r" pod "68613646-0636-483c-90cb-e73f7a6bfc8d" (UID: "68613646-0636-483c-90cb-e73f7a6bfc8d"). InnerVolumeSpecName "kube-api-access-tng4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:27:11.206762 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:11.206682    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tng4r\" (UniqueName: \"kubernetes.io/projected/68613646-0636-483c-90cb-e73f7a6bfc8d-kube-api-access-tng4r\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:11.206762 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:11.206713    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/68613646-0636-483c-90cb-e73f7a6bfc8d-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:11.943057 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:11.943023    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"
Oct 30 22:27:11.943330 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:11.943060    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm" event={"ID":"68613646-0636-483c-90cb-e73f7a6bfc8d","Type":"ContainerDied","Data":"0f39a9cf3821b8324b98112f322d9a2d9392212cd23b181f67bc5ccbc52a0dc8"}
Oct 30 22:27:11.943330 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:11.943109    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f39a9cf3821b8324b98112f322d9a2d9392212cd23b181f67bc5ccbc52a0dc8"
Oct 30 22:27:15.054267 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:15.054214    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"]
Oct 30 22:27:15.104189 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:15.104158    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-87gz6"]
Oct 30 22:27:15.446357 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:15.446326    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc23a315-9191-4cd0-8cf4-4ffb188b2109" path="/var/lib/kubelet/pods/bc23a315-9191-4cd0-8cf4-4ffb188b2109/volumes"
Oct 30 22:27:18.361482 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:27:18.361445    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:27:21.319294 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:21.319259    5586 scope.go:122] "RemoveContainer" containerID="e6b99867f20085f2d9cfe34e8639a2ab9b4418393600427a39afdae2b3733a14"
Oct 30 22:27:21.329002 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:21.328977    5586 scope.go:122] "RemoveContainer" containerID="3c61d7c4c5c26b1f4b36d6f90dec281d246e585ea0ac51c2ba393a8e375944d5"
Oct 30 22:27:34.010181 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:34.010140    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:48.115126 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.115093    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"]
Oct 30 22:27:48.115573 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.115525    5586 cpu_manager.go:414] "containerMap: removing container" podUID="68613646-0636-483c-90cb-e73f7a6bfc8d" containerName="revoke-keys"
Oct 30 22:27:48.115573 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.115536    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68613646-0636-483c-90cb-e73f7a6bfc8d" containerName="revoke-keys"
Oct 30 22:27:48.115656 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.115624    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="68613646-0636-483c-90cb-e73f7a6bfc8d" containerName="revoke-keys"
Oct 30 22:27:48.119846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.119826    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"
Oct 30 22:27:48.210332 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.210302    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:27:48.210519 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.210393    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:27:48.210519 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.210417    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:27:48.212761 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.212738    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"]
Oct 30 22:27:48.221484 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.221470    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:27:48.236513 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.236483    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/583359e2-b5c3-48a7-b673-e8eda96b493b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-fczj8\" (UID: \"583359e2-b5c3-48a7-b673-e8eda96b493b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"
Oct 30 22:27:48.236655 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.236528    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/583359e2-b5c3-48a7-b673-e8eda96b493b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-fczj8\" (UID: \"583359e2-b5c3-48a7-b673-e8eda96b493b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"
Oct 30 22:27:48.236655 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.236604    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmqr9\" (UniqueName: \"kubernetes.io/projected/583359e2-b5c3-48a7-b673-e8eda96b493b-kube-api-access-jmqr9\") pod \"maas-api-revoke-sub-0b7e8edb-fczj8\" (UID: \"583359e2-b5c3-48a7-b673-e8eda96b493b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"
Oct 30 22:27:48.337488 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.337456    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmqr9\" (UniqueName: \"kubernetes.io/projected/583359e2-b5c3-48a7-b673-e8eda96b493b-kube-api-access-jmqr9\") pod \"maas-api-revoke-sub-0b7e8edb-fczj8\" (UID: \"583359e2-b5c3-48a7-b673-e8eda96b493b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"
Oct 30 22:27:48.337671 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.337571    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/583359e2-b5c3-48a7-b673-e8eda96b493b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-fczj8\" (UID: \"583359e2-b5c3-48a7-b673-e8eda96b493b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"
Oct 30 22:27:48.337671 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.337627    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/583359e2-b5c3-48a7-b673-e8eda96b493b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-fczj8\" (UID: \"583359e2-b5c3-48a7-b673-e8eda96b493b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"
Oct 30 22:27:48.338312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.338286    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/583359e2-b5c3-48a7-b673-e8eda96b493b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-fczj8\" (UID: \"583359e2-b5c3-48a7-b673-e8eda96b493b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"
Oct 30 22:27:48.339942 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.339912    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/583359e2-b5c3-48a7-b673-e8eda96b493b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-fczj8\" (UID: \"583359e2-b5c3-48a7-b673-e8eda96b493b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"
Oct 30 22:27:48.361322 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:27:48.361286    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:27:48.394175 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.394105    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmqr9\" (UniqueName: \"kubernetes.io/projected/583359e2-b5c3-48a7-b673-e8eda96b493b-kube-api-access-jmqr9\") pod \"maas-api-revoke-sub-0b7e8edb-fczj8\" (UID: \"583359e2-b5c3-48a7-b673-e8eda96b493b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"
Oct 30 22:27:48.433510 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.433479    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"
Oct 30 22:27:48.806867 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:48.806841    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"]
Oct 30 22:27:48.808782 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:27:48.808752    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod583359e2_b5c3_48a7_b673_e8eda96b493b.slice/crio-501f006bdf7b07590ccc8b6f829a02faa86319052ad9160f3f30e93d6844c0a5 WatchSource:0}: Error finding container 501f006bdf7b07590ccc8b6f829a02faa86319052ad9160f3f30e93d6844c0a5: Status 404 returned error can't find the container with id 501f006bdf7b07590ccc8b6f829a02faa86319052ad9160f3f30e93d6844c0a5
Oct 30 22:27:49.179250 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:49.179203    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8" event={"ID":"583359e2-b5c3-48a7-b673-e8eda96b493b","Type":"ContainerStarted","Data":"cb1f7c3bd53e67225f37e007b52a34d065ea13bc9f222de482b95089838c2d63"}
Oct 30 22:27:49.179652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:49.179290    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8" event={"ID":"583359e2-b5c3-48a7-b673-e8eda96b493b","Type":"ContainerStarted","Data":"501f006bdf7b07590ccc8b6f829a02faa86319052ad9160f3f30e93d6844c0a5"}
Oct 30 22:27:49.299733 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:49.299675    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8" podStartSLOduration=1.299660483 podStartE2EDuration="1.299660483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:27:49.297385773 +0000 UTC m=+3211.969289708" watchObservedRunningTime="2026-10-30 22:27:49.299660483 +0000 UTC m=+3211.971564418"
Oct 30 22:27:50.186463 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:50.186420    5586 generic.go:356] "Generic (PLEG): container finished" podID="583359e2-b5c3-48a7-b673-e8eda96b493b" containerID="cb1f7c3bd53e67225f37e007b52a34d065ea13bc9f222de482b95089838c2d63" exitCode=0
Oct 30 22:27:50.186902 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:50.186509    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8" event={"ID":"583359e2-b5c3-48a7-b673-e8eda96b493b","Type":"ContainerDied","Data":"cb1f7c3bd53e67225f37e007b52a34d065ea13bc9f222de482b95089838c2d63"}
Oct 30 22:27:51.327143 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:51.327115    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"
Oct 30 22:27:51.366094 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:51.366058    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/583359e2-b5c3-48a7-b673-e8eda96b493b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/583359e2-b5c3-48a7-b673-e8eda96b493b-maas-api-openshift-ca\") pod \"583359e2-b5c3-48a7-b673-e8eda96b493b\" (UID: \"583359e2-b5c3-48a7-b673-e8eda96b493b\") "
Oct 30 22:27:51.366312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:51.366134    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/583359e2-b5c3-48a7-b673-e8eda96b493b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/583359e2-b5c3-48a7-b673-e8eda96b493b-maas-api-xks-ca\") pod \"583359e2-b5c3-48a7-b673-e8eda96b493b\" (UID: \"583359e2-b5c3-48a7-b673-e8eda96b493b\") "
Oct 30 22:27:51.366312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:51.366268    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/583359e2-b5c3-48a7-b673-e8eda96b493b-kube-api-access-jmqr9\" (UniqueName: \"kubernetes.io/projected/583359e2-b5c3-48a7-b673-e8eda96b493b-kube-api-access-jmqr9\") pod \"583359e2-b5c3-48a7-b673-e8eda96b493b\" (UID: \"583359e2-b5c3-48a7-b673-e8eda96b493b\") "
Oct 30 22:27:51.366541 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:51.366515    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/583359e2-b5c3-48a7-b673-e8eda96b493b-maas-api-openshift-ca" pod "583359e2-b5c3-48a7-b673-e8eda96b493b" (UID: "583359e2-b5c3-48a7-b673-e8eda96b493b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:27:51.368327 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:51.368296    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/583359e2-b5c3-48a7-b673-e8eda96b493b-maas-api-xks-ca" pod "583359e2-b5c3-48a7-b673-e8eda96b493b" (UID: "583359e2-b5c3-48a7-b673-e8eda96b493b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:27:51.368432 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:51.368377    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/583359e2-b5c3-48a7-b673-e8eda96b493b-kube-api-access-jmqr9" pod "583359e2-b5c3-48a7-b673-e8eda96b493b" (UID: "583359e2-b5c3-48a7-b673-e8eda96b493b"). InnerVolumeSpecName "kube-api-access-jmqr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:27:51.467942 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:51.467911    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/583359e2-b5c3-48a7-b673-e8eda96b493b-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:51.467942 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:51.467940    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmqr9\" (UniqueName: \"kubernetes.io/projected/583359e2-b5c3-48a7-b673-e8eda96b493b-kube-api-access-jmqr9\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:51.468137 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:51.467950    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/583359e2-b5c3-48a7-b673-e8eda96b493b-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:27:52.199223 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:52.199197    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"
Oct 30 22:27:52.199424 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:52.199262    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8" event={"ID":"583359e2-b5c3-48a7-b673-e8eda96b493b","Type":"ContainerDied","Data":"501f006bdf7b07590ccc8b6f829a02faa86319052ad9160f3f30e93d6844c0a5"}
Oct 30 22:27:52.199424 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:52.199316    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="501f006bdf7b07590ccc8b6f829a02faa86319052ad9160f3f30e93d6844c0a5"
Oct 30 22:27:59.485436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.485391    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"]
Oct 30 22:27:59.485898 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.485806    5586 cpu_manager.go:414] "containerMap: removing container" podUID="583359e2-b5c3-48a7-b673-e8eda96b493b" containerName="revoke-keys"
Oct 30 22:27:59.485898 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.485817    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="583359e2-b5c3-48a7-b673-e8eda96b493b" containerName="revoke-keys"
Oct 30 22:27:59.486095 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.485936    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="583359e2-b5c3-48a7-b673-e8eda96b493b" containerName="revoke-keys"
Oct 30 22:27:59.489365 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.489341    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"
Oct 30 22:27:59.545888 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.545850    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"]
Oct 30 22:27:59.551331 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.551296    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd9f5cbb-0393-4359-96ab-403ac5311405-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-p6ft4\" (UID: \"bd9f5cbb-0393-4359-96ab-403ac5311405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"
Oct 30 22:27:59.551479 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.551366    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkh4h\" (UniqueName: \"kubernetes.io/projected/bd9f5cbb-0393-4359-96ab-403ac5311405-kube-api-access-zkh4h\") pod \"maas-api-revoke-sub-3d4156f1-p6ft4\" (UID: \"bd9f5cbb-0393-4359-96ab-403ac5311405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"
Oct 30 22:27:59.551604 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.551580    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd9f5cbb-0393-4359-96ab-403ac5311405-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-p6ft4\" (UID: \"bd9f5cbb-0393-4359-96ab-403ac5311405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"
Oct 30 22:27:59.592107 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.592064    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:27:59.592107 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.592077    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:27:59.592369 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.592118    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:27:59.592369 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.592219    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:27:59.652809 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.652769    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd9f5cbb-0393-4359-96ab-403ac5311405-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-p6ft4\" (UID: \"bd9f5cbb-0393-4359-96ab-403ac5311405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"
Oct 30 22:27:59.653039 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.652827    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkh4h\" (UniqueName: \"kubernetes.io/projected/bd9f5cbb-0393-4359-96ab-403ac5311405-kube-api-access-zkh4h\") pod \"maas-api-revoke-sub-3d4156f1-p6ft4\" (UID: \"bd9f5cbb-0393-4359-96ab-403ac5311405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"
Oct 30 22:27:59.653039 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.652914    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd9f5cbb-0393-4359-96ab-403ac5311405-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-p6ft4\" (UID: \"bd9f5cbb-0393-4359-96ab-403ac5311405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"
Oct 30 22:27:59.653570 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.653539    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd9f5cbb-0393-4359-96ab-403ac5311405-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-p6ft4\" (UID: \"bd9f5cbb-0393-4359-96ab-403ac5311405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"
Oct 30 22:27:59.655433 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.655404    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd9f5cbb-0393-4359-96ab-403ac5311405-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-p6ft4\" (UID: \"bd9f5cbb-0393-4359-96ab-403ac5311405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"
Oct 30 22:27:59.709658 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.709621    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkh4h\" (UniqueName: \"kubernetes.io/projected/bd9f5cbb-0393-4359-96ab-403ac5311405-kube-api-access-zkh4h\") pod \"maas-api-revoke-sub-3d4156f1-p6ft4\" (UID: \"bd9f5cbb-0393-4359-96ab-403ac5311405\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"
Oct 30 22:27:59.802055 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:27:59.801964    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"
Oct 30 22:28:00.175621 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:00.175416    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"]
Oct 30 22:28:00.178098 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:28:00.178070    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd9f5cbb_0393_4359_96ab_403ac5311405.slice/crio-6f35fe68d44e8476125631ca16d1905af239fe43443a0fb3a6d37ca171d61193 WatchSource:0}: Error finding container 6f35fe68d44e8476125631ca16d1905af239fe43443a0fb3a6d37ca171d61193: Status 404 returned error can't find the container with id 6f35fe68d44e8476125631ca16d1905af239fe43443a0fb3a6d37ca171d61193
Oct 30 22:28:00.247882 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:00.247844    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4" event={"ID":"bd9f5cbb-0393-4359-96ab-403ac5311405","Type":"ContainerStarted","Data":"6f35fe68d44e8476125631ca16d1905af239fe43443a0fb3a6d37ca171d61193"}
Oct 30 22:28:01.258302 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:01.258264    5586 generic.go:356] "Generic (PLEG): container finished" podID="bd9f5cbb-0393-4359-96ab-403ac5311405" containerID="8786359d7ca0576be09649f41fff05cef6c2c27ed06cb2d4cf4b50333ddbb2e7" exitCode=0
Oct 30 22:28:01.259077 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:01.258335    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4" event={"ID":"bd9f5cbb-0393-4359-96ab-403ac5311405","Type":"ContainerDied","Data":"8786359d7ca0576be09649f41fff05cef6c2c27ed06cb2d4cf4b50333ddbb2e7"}
Oct 30 22:28:02.996865 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:02.996833    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"
Oct 30 22:28:03.189994 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:03.189890    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd9f5cbb-0393-4359-96ab-403ac5311405-kube-api-access-zkh4h\" (UniqueName: \"kubernetes.io/projected/bd9f5cbb-0393-4359-96ab-403ac5311405-kube-api-access-zkh4h\") pod \"bd9f5cbb-0393-4359-96ab-403ac5311405\" (UID: \"bd9f5cbb-0393-4359-96ab-403ac5311405\") "
Oct 30 22:28:03.191655 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:03.191627    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd9f5cbb-0393-4359-96ab-403ac5311405-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd9f5cbb-0393-4359-96ab-403ac5311405-maas-api-openshift-ca\") pod \"bd9f5cbb-0393-4359-96ab-403ac5311405\" (UID: \"bd9f5cbb-0393-4359-96ab-403ac5311405\") "
Oct 30 22:28:03.192418 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:03.192394    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd9f5cbb-0393-4359-96ab-403ac5311405-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd9f5cbb-0393-4359-96ab-403ac5311405-maas-api-xks-ca\") pod \"bd9f5cbb-0393-4359-96ab-403ac5311405\" (UID: \"bd9f5cbb-0393-4359-96ab-403ac5311405\") "
Oct 30 22:28:03.193180 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:03.192177    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd9f5cbb-0393-4359-96ab-403ac5311405-maas-api-openshift-ca" pod "bd9f5cbb-0393-4359-96ab-403ac5311405" (UID: "bd9f5cbb-0393-4359-96ab-403ac5311405"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:28:03.212794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:03.212747    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd9f5cbb-0393-4359-96ab-403ac5311405-maas-api-xks-ca" pod "bd9f5cbb-0393-4359-96ab-403ac5311405" (UID: "bd9f5cbb-0393-4359-96ab-403ac5311405"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:28:03.216119 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:03.216080    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd9f5cbb-0393-4359-96ab-403ac5311405-kube-api-access-zkh4h" pod "bd9f5cbb-0393-4359-96ab-403ac5311405" (UID: "bd9f5cbb-0393-4359-96ab-403ac5311405"). InnerVolumeSpecName "kube-api-access-zkh4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:03.294246 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:03.294194    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bd9f5cbb-0393-4359-96ab-403ac5311405-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:03.294246 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:03.294225    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bd9f5cbb-0393-4359-96ab-403ac5311405-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:03.294246 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:03.294253    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkh4h\" (UniqueName: \"kubernetes.io/projected/bd9f5cbb-0393-4359-96ab-403ac5311405-kube-api-access-zkh4h\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:03.302638 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:03.302597    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4" event={"ID":"bd9f5cbb-0393-4359-96ab-403ac5311405","Type":"ContainerDied","Data":"6f35fe68d44e8476125631ca16d1905af239fe43443a0fb3a6d37ca171d61193"}
Oct 30 22:28:03.302855 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:03.302639    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"
Oct 30 22:28:03.302855 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:03.302687    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f35fe68d44e8476125631ca16d1905af239fe43443a0fb3a6d37ca171d61193"
Oct 30 22:28:18.360971 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:28:18.360933    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:28:21.356696 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:21.356661    5586 scope.go:122] "RemoveContainer" containerID="7cc4b4bf3fdea4c383f7941d52ad7855b3cc39e30174c63f20a5bc9de4ce003d"
Oct 30 22:28:32.231286 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.231126    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"]
Oct 30 22:28:32.281921 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.281885    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-p6ft4"]
Oct 30 22:28:32.352960 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.352918    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"]
Oct 30 22:28:32.353489 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.353467    5586 cpu_manager.go:414] "containerMap: removing container" podUID="bd9f5cbb-0393-4359-96ab-403ac5311405" containerName="revoke-keys"
Oct 30 22:28:32.353489 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.353488    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd9f5cbb-0393-4359-96ab-403ac5311405" containerName="revoke-keys"
Oct 30 22:28:32.353678 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.353626    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd9f5cbb-0393-4359-96ab-403ac5311405" containerName="revoke-keys"
Oct 30 22:28:32.357981 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.357960    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"
Oct 30 22:28:32.412190 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.412153    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/66d9d080-3d19-4a87-ae78-dae686f8c7bc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-g4vg4\" (UID: \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"
Oct 30 22:28:32.412393 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.412202    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w769n\" (UniqueName: \"kubernetes.io/projected/66d9d080-3d19-4a87-ae78-dae686f8c7bc-kube-api-access-w769n\") pod \"maas-api-revoke-sub-3d4156f1-g4vg4\" (UID: \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"
Oct 30 22:28:32.412393 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.412330    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/66d9d080-3d19-4a87-ae78-dae686f8c7bc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-g4vg4\" (UID: \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"
Oct 30 22:28:32.413148 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.412972    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:28:32.413570 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.413551    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:28:32.413736 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.413575    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:28:32.416796 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.416775    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"]
Oct 30 22:28:32.462177 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.462144    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:28:32.513570 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.513463    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/66d9d080-3d19-4a87-ae78-dae686f8c7bc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-g4vg4\" (UID: \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"
Oct 30 22:28:32.513570 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.513532    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w769n\" (UniqueName: \"kubernetes.io/projected/66d9d080-3d19-4a87-ae78-dae686f8c7bc-kube-api-access-w769n\") pod \"maas-api-revoke-sub-3d4156f1-g4vg4\" (UID: \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"
Oct 30 22:28:32.513570 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.513558    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/66d9d080-3d19-4a87-ae78-dae686f8c7bc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-g4vg4\" (UID: \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"
Oct 30 22:28:32.514376 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.514352    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/66d9d080-3d19-4a87-ae78-dae686f8c7bc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-g4vg4\" (UID: \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"
Oct 30 22:28:32.516053 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.516030    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/66d9d080-3d19-4a87-ae78-dae686f8c7bc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-g4vg4\" (UID: \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"
Oct 30 22:28:32.571266 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.571215    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w769n\" (UniqueName: \"kubernetes.io/projected/66d9d080-3d19-4a87-ae78-dae686f8c7bc-kube-api-access-w769n\") pod \"maas-api-revoke-sub-3d4156f1-g4vg4\" (UID: \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"
Oct 30 22:28:32.671087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:32.671033    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"
Oct 30 22:28:33.044113 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:33.044086    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"]
Oct 30 22:28:33.046605 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:28:33.046569    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66d9d080_3d19_4a87_ae78_dae686f8c7bc.slice/crio-71024a13b174bde923a327d32f5b5d0fc176cf34f992fc530e36896151031995 WatchSource:0}: Error finding container 71024a13b174bde923a327d32f5b5d0fc176cf34f992fc530e36896151031995: Status 404 returned error can't find the container with id 71024a13b174bde923a327d32f5b5d0fc176cf34f992fc530e36896151031995
Oct 30 22:28:33.047185 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:33.047159    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"]
Oct 30 22:28:33.051093 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:33.051071    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-2zmnm"]
Oct 30 22:28:33.446473 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:33.446434    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd9f5cbb-0393-4359-96ab-403ac5311405" path="/var/lib/kubelet/pods/bd9f5cbb-0393-4359-96ab-403ac5311405/volumes"
Oct 30 22:28:33.446887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:33.446877    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa31c3d5-99bf-4fea-9601-f643a0c690d5" path="/var/lib/kubelet/pods/fa31c3d5-99bf-4fea-9601-f643a0c690d5/volumes"
Oct 30 22:28:33.504336 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:33.504298    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4" event={"ID":"66d9d080-3d19-4a87-ae78-dae686f8c7bc","Type":"ContainerStarted","Data":"b27e8d9111f97c801e41a801e247680d33b6cd33d4c060fff08b5bb267706a29"}
Oct 30 22:28:33.504532 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:33.504352    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4" event={"ID":"66d9d080-3d19-4a87-ae78-dae686f8c7bc","Type":"ContainerStarted","Data":"71024a13b174bde923a327d32f5b5d0fc176cf34f992fc530e36896151031995"}
Oct 30 22:28:33.620820 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:33.620707    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4" podStartSLOduration=1.6206915240000002 podStartE2EDuration="1.620691524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:33.619585992 +0000 UTC m=+3256.291489929" watchObservedRunningTime="2026-10-30 22:28:33.620691524 +0000 UTC m=+3256.292595459"
Oct 30 22:28:34.367507 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:34.367468    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:34.512394 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:34.512360    5586 generic.go:356] "Generic (PLEG): container finished" podID="66d9d080-3d19-4a87-ae78-dae686f8c7bc" containerID="b27e8d9111f97c801e41a801e247680d33b6cd33d4c060fff08b5bb267706a29" exitCode=0
Oct 30 22:28:34.512794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:34.512408    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4" event={"ID":"66d9d080-3d19-4a87-ae78-dae686f8c7bc","Type":"ContainerDied","Data":"b27e8d9111f97c801e41a801e247680d33b6cd33d4c060fff08b5bb267706a29"}
Oct 30 22:28:35.669258 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:35.669216    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"
Oct 30 22:28:35.740105 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:35.740068    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/66d9d080-3d19-4a87-ae78-dae686f8c7bc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/66d9d080-3d19-4a87-ae78-dae686f8c7bc-maas-api-openshift-ca\") pod \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\" (UID: \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\") "
Oct 30 22:28:35.740331 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:35.740124    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66d9d080-3d19-4a87-ae78-dae686f8c7bc-kube-api-access-w769n\" (UniqueName: \"kubernetes.io/projected/66d9d080-3d19-4a87-ae78-dae686f8c7bc-kube-api-access-w769n\") pod \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\" (UID: \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\") "
Oct 30 22:28:35.740331 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:35.740183    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/66d9d080-3d19-4a87-ae78-dae686f8c7bc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/66d9d080-3d19-4a87-ae78-dae686f8c7bc-maas-api-xks-ca\") pod \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\" (UID: \"66d9d080-3d19-4a87-ae78-dae686f8c7bc\") "
Oct 30 22:28:35.740591 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:35.740562    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66d9d080-3d19-4a87-ae78-dae686f8c7bc-maas-api-openshift-ca" pod "66d9d080-3d19-4a87-ae78-dae686f8c7bc" (UID: "66d9d080-3d19-4a87-ae78-dae686f8c7bc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:28:35.742401 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:35.742371    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66d9d080-3d19-4a87-ae78-dae686f8c7bc-maas-api-xks-ca" pod "66d9d080-3d19-4a87-ae78-dae686f8c7bc" (UID: "66d9d080-3d19-4a87-ae78-dae686f8c7bc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:28:35.742401 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:35.742398    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66d9d080-3d19-4a87-ae78-dae686f8c7bc-kube-api-access-w769n" pod "66d9d080-3d19-4a87-ae78-dae686f8c7bc" (UID: "66d9d080-3d19-4a87-ae78-dae686f8c7bc"). InnerVolumeSpecName "kube-api-access-w769n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:35.841339 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:35.841212    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w769n\" (UniqueName: \"kubernetes.io/projected/66d9d080-3d19-4a87-ae78-dae686f8c7bc-kube-api-access-w769n\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:35.841339 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:35.841284    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/66d9d080-3d19-4a87-ae78-dae686f8c7bc-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:35.841339 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:35.841294    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/66d9d080-3d19-4a87-ae78-dae686f8c7bc-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:28:36.526807 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:36.526770    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"
Oct 30 22:28:36.526807 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:36.526795    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4" event={"ID":"66d9d080-3d19-4a87-ae78-dae686f8c7bc","Type":"ContainerDied","Data":"71024a13b174bde923a327d32f5b5d0fc176cf34f992fc530e36896151031995"}
Oct 30 22:28:36.527045 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:28:36.526850    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71024a13b174bde923a327d32f5b5d0fc176cf34f992fc530e36896151031995"
Oct 30 22:28:48.360942 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:28:48.360899    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:29:02.051471 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:02.051426    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"]
Oct 30 22:29:02.100215 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:02.100160    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-9l85z"]
Oct 30 22:29:03.446343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:03.446303    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ad30631-2b97-4c5b-b220-de754ab095ba" path="/var/lib/kubelet/pods/7ad30631-2b97-4c5b-b220-de754ab095ba/volumes"
Oct 30 22:29:14.039390 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.039353    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"]
Oct 30 22:29:14.039819 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.039805    5586 cpu_manager.go:414] "containerMap: removing container" podUID="66d9d080-3d19-4a87-ae78-dae686f8c7bc" containerName="revoke-keys"
Oct 30 22:29:14.039864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.039820    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66d9d080-3d19-4a87-ae78-dae686f8c7bc" containerName="revoke-keys"
Oct 30 22:29:14.039931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.039921    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="66d9d080-3d19-4a87-ae78-dae686f8c7bc" containerName="revoke-keys"
Oct 30 22:29:14.043419 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.043403    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"
Oct 30 22:29:14.096423 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.096377    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:29:14.096620 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.096439    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:29:14.145555 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.145519    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:29:14.145555 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.145519    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:29:14.148644 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.148620    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"]
Oct 30 22:29:14.154123 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.154088    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-5nl8t\" (UID: \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"
Oct 30 22:29:14.154243 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.154211    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-5nl8t\" (UID: \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"
Oct 30 22:29:14.154294 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.154284    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6csdq\" (UniqueName: \"kubernetes.io/projected/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-kube-api-access-6csdq\") pod \"maas-api-revoke-sub-e2fc9c5b-5nl8t\" (UID: \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"
Oct 30 22:29:14.255181 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.255142    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6csdq\" (UniqueName: \"kubernetes.io/projected/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-kube-api-access-6csdq\") pod \"maas-api-revoke-sub-e2fc9c5b-5nl8t\" (UID: \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"
Oct 30 22:29:14.255401 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.255221    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-5nl8t\" (UID: \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"
Oct 30 22:29:14.255401 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.255302    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-5nl8t\" (UID: \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"
Oct 30 22:29:14.255906 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.255882    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-5nl8t\" (UID: \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"
Oct 30 22:29:14.257713 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.257685    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-5nl8t\" (UID: \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"
Oct 30 22:29:14.311968 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.311895    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6csdq\" (UniqueName: \"kubernetes.io/projected/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-kube-api-access-6csdq\") pod \"maas-api-revoke-sub-e2fc9c5b-5nl8t\" (UID: \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"
Oct 30 22:29:14.355879 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.355841    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"
Oct 30 22:29:14.735096 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.735069    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"]
Oct 30 22:29:14.737391 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:29:14.737365    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a5d19a2_e6aa_48ec_9f41_0a0670268e0b.slice/crio-2e7d564d6d582a7a57003acf0e955c7f0740ee2292f2c80ecb4fdbef77cd70fb WatchSource:0}: Error finding container 2e7d564d6d582a7a57003acf0e955c7f0740ee2292f2c80ecb4fdbef77cd70fb: Status 404 returned error can't find the container with id 2e7d564d6d582a7a57003acf0e955c7f0740ee2292f2c80ecb4fdbef77cd70fb
Oct 30 22:29:14.786426 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:14.786390    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t" event={"ID":"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b","Type":"ContainerStarted","Data":"2e7d564d6d582a7a57003acf0e955c7f0740ee2292f2c80ecb4fdbef77cd70fb"}
Oct 30 22:29:15.044263 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:15.044210    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"]
Oct 30 22:29:15.046469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:15.046447    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-lsvj6"]
Oct 30 22:29:15.446629 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:15.446591    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cbe5ecef-5839-4740-9b27-2de61f2b0b1c" path="/var/lib/kubelet/pods/cbe5ecef-5839-4740-9b27-2de61f2b0b1c/volumes"
Oct 30 22:29:15.794026 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:15.793940    5586 generic.go:356] "Generic (PLEG): container finished" podID="7a5d19a2-e6aa-48ec-9f41-0a0670268e0b" containerID="748934ee75c2486beb75cf0de5bc6f16c97b601d0bf745c35c4f6d96ed10553b" exitCode=0
Oct 30 22:29:15.794026 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:15.793984    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t" event={"ID":"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b","Type":"ContainerDied","Data":"748934ee75c2486beb75cf0de5bc6f16c97b601d0bf745c35c4f6d96ed10553b"}
Oct 30 22:29:16.940633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:16.940604    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"
Oct 30 22:29:17.083885 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:17.083790    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-maas-api-xks-ca\") pod \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\" (UID: \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\") "
Oct 30 22:29:17.083885 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:17.083865    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-kube-api-access-6csdq\" (UniqueName: \"kubernetes.io/projected/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-kube-api-access-6csdq\") pod \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\" (UID: \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\") "
Oct 30 22:29:17.084138 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:17.084051    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-maas-api-openshift-ca\") pod \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\" (UID: \"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b\") "
Oct 30 22:29:17.084425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:17.084395    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-maas-api-openshift-ca" pod "7a5d19a2-e6aa-48ec-9f41-0a0670268e0b" (UID: "7a5d19a2-e6aa-48ec-9f41-0a0670268e0b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:17.086016 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:17.085990    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-maas-api-xks-ca" pod "7a5d19a2-e6aa-48ec-9f41-0a0670268e0b" (UID: "7a5d19a2-e6aa-48ec-9f41-0a0670268e0b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:29:17.086280 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:17.086253    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-kube-api-access-6csdq" pod "7a5d19a2-e6aa-48ec-9f41-0a0670268e0b" (UID: "7a5d19a2-e6aa-48ec-9f41-0a0670268e0b"). InnerVolumeSpecName "kube-api-access-6csdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:17.185503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:17.185465    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.185503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:17.185499    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.185503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:17.185508    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6csdq\" (UniqueName: \"kubernetes.io/projected/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b-kube-api-access-6csdq\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:17.807342 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:17.807257    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"
Oct 30 22:29:17.807523 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:17.807255    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t" event={"ID":"7a5d19a2-e6aa-48ec-9f41-0a0670268e0b","Type":"ContainerDied","Data":"2e7d564d6d582a7a57003acf0e955c7f0740ee2292f2c80ecb4fdbef77cd70fb"}
Oct 30 22:29:17.807523 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:17.807393    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e7d564d6d582a7a57003acf0e955c7f0740ee2292f2c80ecb4fdbef77cd70fb"
Oct 30 22:29:18.125987 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:18.125907    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:29:18.131281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:18.131225    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:29:18.135116 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:18.135089    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:29:18.140459 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:18.140434    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:29:18.361392 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:29:18.361356    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:29:21.766897 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:21.766851    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal" podUID="6ea3e08f52bca920602c76ad35bcaf69" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Oct 30 22:29:24.899436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:24.899399    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"]
Oct 30 22:29:24.899859 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:24.899846    5586 cpu_manager.go:414] "containerMap: removing container" podUID="7a5d19a2-e6aa-48ec-9f41-0a0670268e0b" containerName="revoke-keys"
Oct 30 22:29:24.899904 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:24.899859    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a5d19a2-e6aa-48ec-9f41-0a0670268e0b" containerName="revoke-keys"
Oct 30 22:29:24.899961 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:24.899952    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a5d19a2-e6aa-48ec-9f41-0a0670268e0b" containerName="revoke-keys"
Oct 30 22:29:24.903355 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:24.903331    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"
Oct 30 22:29:24.956553 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:24.956510    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a29efb3-d5lc7\" (UID: \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"
Oct 30 22:29:24.956793 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:24.956563    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plq6p\" (UniqueName: \"kubernetes.io/projected/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-kube-api-access-plq6p\") pod \"maas-api-revoke-sub-5a29efb3-d5lc7\" (UID: \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"
Oct 30 22:29:24.956887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:24.956805    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a29efb3-d5lc7\" (UID: \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"
Oct 30 22:29:24.956954 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:24.956906    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:29:24.960955 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:24.960927    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"]
Oct 30 22:29:25.006597 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:25.006558    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:29:25.007007 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:25.006989    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:29:25.007124 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:25.007106    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:29:25.057688 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:25.057644    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a29efb3-d5lc7\" (UID: \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"
Oct 30 22:29:25.057843 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:25.057735    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a29efb3-d5lc7\" (UID: \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"
Oct 30 22:29:25.057843 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:25.057764    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plq6p\" (UniqueName: \"kubernetes.io/projected/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-kube-api-access-plq6p\") pod \"maas-api-revoke-sub-5a29efb3-d5lc7\" (UID: \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"
Oct 30 22:29:25.058532 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:25.058503    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a29efb3-d5lc7\" (UID: \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"
Oct 30 22:29:25.060126 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:25.060108    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a29efb3-d5lc7\" (UID: \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"
Oct 30 22:29:25.115126 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:25.115095    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plq6p\" (UniqueName: \"kubernetes.io/projected/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-kube-api-access-plq6p\") pod \"maas-api-revoke-sub-5a29efb3-d5lc7\" (UID: \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"
Oct 30 22:29:25.216545 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:25.216506    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"
Oct 30 22:29:25.593013 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:25.592984    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"]
Oct 30 22:29:25.595964 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:29:25.595927    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda03eeb81_ee85_422b_bc49_a8cc1c9aa8a6.slice/crio-1b2df81db891d81304a822f6afed6f539929f8b0afc068bff404aa9ae1d69ce3 WatchSource:0}: Error finding container 1b2df81db891d81304a822f6afed6f539929f8b0afc068bff404aa9ae1d69ce3: Status 404 returned error can't find the container with id 1b2df81db891d81304a822f6afed6f539929f8b0afc068bff404aa9ae1d69ce3
Oct 30 22:29:25.859772 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:25.859731    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7" event={"ID":"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6","Type":"ContainerStarted","Data":"1b2df81db891d81304a822f6afed6f539929f8b0afc068bff404aa9ae1d69ce3"}
Oct 30 22:29:26.867441 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:26.867355    5586 generic.go:356] "Generic (PLEG): container finished" podID="a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6" containerID="4fcca0165c5893a257ca27a1eb5f516a8e220be5a611b5edd6018ad0545bd781" exitCode=0
Oct 30 22:29:26.867816 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:26.867431    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7" event={"ID":"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6","Type":"ContainerDied","Data":"4fcca0165c5893a257ca27a1eb5f516a8e220be5a611b5edd6018ad0545bd781"}
Oct 30 22:29:28.064123 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:28.064087    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"
Oct 30 22:29:28.088955 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:28.088915    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-maas-api-openshift-ca\") pod \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\" (UID: \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\") "
Oct 30 22:29:28.089171 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:28.089049    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-kube-api-access-plq6p\" (UniqueName: \"kubernetes.io/projected/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-kube-api-access-plq6p\") pod \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\" (UID: \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\") "
Oct 30 22:29:28.089171 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:28.089127    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-maas-api-xks-ca\") pod \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\" (UID: \"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6\") "
Oct 30 22:29:28.089680 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:28.089648    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-maas-api-openshift-ca" pod "a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6" (UID: "a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:28.091919 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:28.091891    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-kube-api-access-plq6p" pod "a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6" (UID: "a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6"). InnerVolumeSpecName "kube-api-access-plq6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:28.091919 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:28.091902    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-maas-api-xks-ca" pod "a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6" (UID: "a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:29:28.191261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:28.190612    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:28.191261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:28.190650    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plq6p\" (UniqueName: \"kubernetes.io/projected/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-kube-api-access-plq6p\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:28.191261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:28.190669    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:28.910279 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:28.908286    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"
Oct 30 22:29:28.918049 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:28.917840    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7" event={"ID":"a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6","Type":"ContainerDied","Data":"1b2df81db891d81304a822f6afed6f539929f8b0afc068bff404aa9ae1d69ce3"}
Oct 30 22:29:28.918049 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:28.917911    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b2df81db891d81304a822f6afed6f539929f8b0afc068bff404aa9ae1d69ce3"
Oct 30 22:29:34.648186 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:34.648147    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:48.360760 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:29:48.360645    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:29:52.088229 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.088192    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"]
Oct 30 22:29:52.137532 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.137497    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g4vg4"]
Oct 30 22:29:52.210160 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.210123    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"]
Oct 30 22:29:52.210716 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.210687    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6" containerName="revoke-keys"
Oct 30 22:29:52.210716 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.210714    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6" containerName="revoke-keys"
Oct 30 22:29:52.210861 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.210796    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6" containerName="revoke-keys"
Oct 30 22:29:52.213771 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.213755    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"
Oct 30 22:29:52.267488 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.267453    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:29:52.268282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.267681    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:29:52.268282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.267898    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:29:52.268282 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.268099    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:29:52.270895 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.270870    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"]
Oct 30 22:29:52.380862 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.380772    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fed66b60-0f21-4433-b567-449b3c05043d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-b8b85\" (UID: \"fed66b60-0f21-4433-b567-449b3c05043d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"
Oct 30 22:29:52.381035 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.380867    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9khx\" (UniqueName: \"kubernetes.io/projected/fed66b60-0f21-4433-b567-449b3c05043d-kube-api-access-b9khx\") pod \"maas-api-revoke-sub-3d4156f1-b8b85\" (UID: \"fed66b60-0f21-4433-b567-449b3c05043d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"
Oct 30 22:29:52.381035 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.380895    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fed66b60-0f21-4433-b567-449b3c05043d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-b8b85\" (UID: \"fed66b60-0f21-4433-b567-449b3c05043d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"
Oct 30 22:29:52.481517 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.481478    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fed66b60-0f21-4433-b567-449b3c05043d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-b8b85\" (UID: \"fed66b60-0f21-4433-b567-449b3c05043d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"
Oct 30 22:29:52.481721 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.481594    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fed66b60-0f21-4433-b567-449b3c05043d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-b8b85\" (UID: \"fed66b60-0f21-4433-b567-449b3c05043d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"
Oct 30 22:29:52.481721 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.481645    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9khx\" (UniqueName: \"kubernetes.io/projected/fed66b60-0f21-4433-b567-449b3c05043d-kube-api-access-b9khx\") pod \"maas-api-revoke-sub-3d4156f1-b8b85\" (UID: \"fed66b60-0f21-4433-b567-449b3c05043d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"
Oct 30 22:29:52.482295 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.482273    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fed66b60-0f21-4433-b567-449b3c05043d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-b8b85\" (UID: \"fed66b60-0f21-4433-b567-449b3c05043d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"
Oct 30 22:29:52.483917 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.483890    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fed66b60-0f21-4433-b567-449b3c05043d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-b8b85\" (UID: \"fed66b60-0f21-4433-b567-449b3c05043d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"
Oct 30 22:29:52.538024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.537985    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9khx\" (UniqueName: \"kubernetes.io/projected/fed66b60-0f21-4433-b567-449b3c05043d-kube-api-access-b9khx\") pod \"maas-api-revoke-sub-3d4156f1-b8b85\" (UID: \"fed66b60-0f21-4433-b567-449b3c05043d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"
Oct 30 22:29:52.826940 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:52.826905    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"
Oct 30 22:29:53.201669 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:53.201579    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"]
Oct 30 22:29:53.446470 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:53.446437    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66d9d080-3d19-4a87-ae78-dae686f8c7bc" path="/var/lib/kubelet/pods/66d9d080-3d19-4a87-ae78-dae686f8c7bc/volumes"
Oct 30 22:29:54.056899 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:54.056858    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85" event={"ID":"fed66b60-0f21-4433-b567-449b3c05043d","Type":"ContainerStarted","Data":"e78763107f8bffaa314cf2a2390a8320d07f366ca000f5c0db19b122e99a13b0"}
Oct 30 22:29:54.057099 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:54.056917    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85" event={"ID":"fed66b60-0f21-4433-b567-449b3c05043d","Type":"ContainerStarted","Data":"bfe0c7dbd084becf99d59962a0519e671526905b3097b29e5fbdd35c93b2f40f"}
Oct 30 22:29:54.179338 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:54.179271    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85" podStartSLOduration=2.179256413 podStartE2EDuration="2.179256413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:54.178509181 +0000 UTC m=+3336.850413117" watchObservedRunningTime="2026-10-30 22:29:54.179256413 +0000 UTC m=+3336.851160347"
Oct 30 22:29:55.064379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:55.064342    5586 generic.go:356] "Generic (PLEG): container finished" podID="fed66b60-0f21-4433-b567-449b3c05043d" containerID="e78763107f8bffaa314cf2a2390a8320d07f366ca000f5c0db19b122e99a13b0" exitCode=0
Oct 30 22:29:55.064792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:55.064430    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85" event={"ID":"fed66b60-0f21-4433-b567-449b3c05043d","Type":"ContainerDied","Data":"e78763107f8bffaa314cf2a2390a8320d07f366ca000f5c0db19b122e99a13b0"}
Oct 30 22:29:56.208946 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:56.208919    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"
Oct 30 22:29:56.320032 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:56.319997    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fed66b60-0f21-4433-b567-449b3c05043d-kube-api-access-b9khx\" (UniqueName: \"kubernetes.io/projected/fed66b60-0f21-4433-b567-449b3c05043d-kube-api-access-b9khx\") pod \"fed66b60-0f21-4433-b567-449b3c05043d\" (UID: \"fed66b60-0f21-4433-b567-449b3c05043d\") "
Oct 30 22:29:56.320246 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:56.320130    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fed66b60-0f21-4433-b567-449b3c05043d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fed66b60-0f21-4433-b567-449b3c05043d-maas-api-openshift-ca\") pod \"fed66b60-0f21-4433-b567-449b3c05043d\" (UID: \"fed66b60-0f21-4433-b567-449b3c05043d\") "
Oct 30 22:29:56.320246 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:56.320177    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fed66b60-0f21-4433-b567-449b3c05043d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fed66b60-0f21-4433-b567-449b3c05043d-maas-api-xks-ca\") pod \"fed66b60-0f21-4433-b567-449b3c05043d\" (UID: \"fed66b60-0f21-4433-b567-449b3c05043d\") "
Oct 30 22:29:56.320601 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:56.320573    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fed66b60-0f21-4433-b567-449b3c05043d-maas-api-openshift-ca" pod "fed66b60-0f21-4433-b567-449b3c05043d" (UID: "fed66b60-0f21-4433-b567-449b3c05043d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:29:56.322318 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:56.322283    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fed66b60-0f21-4433-b567-449b3c05043d-maas-api-xks-ca" pod "fed66b60-0f21-4433-b567-449b3c05043d" (UID: "fed66b60-0f21-4433-b567-449b3c05043d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:29:56.322318 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:56.322289    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed66b60-0f21-4433-b567-449b3c05043d-kube-api-access-b9khx" pod "fed66b60-0f21-4433-b567-449b3c05043d" (UID: "fed66b60-0f21-4433-b567-449b3c05043d"). InnerVolumeSpecName "kube-api-access-b9khx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:29:56.421950 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:56.421909    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fed66b60-0f21-4433-b567-449b3c05043d-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:56.421950 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:56.421944    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fed66b60-0f21-4433-b567-449b3c05043d-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:56.421950 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:56.421953    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9khx\" (UniqueName: \"kubernetes.io/projected/fed66b60-0f21-4433-b567-449b3c05043d-kube-api-access-b9khx\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:29:57.077572 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:57.077539    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"
Oct 30 22:29:57.077572 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:57.077564    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85" event={"ID":"fed66b60-0f21-4433-b567-449b3c05043d","Type":"ContainerDied","Data":"bfe0c7dbd084becf99d59962a0519e671526905b3097b29e5fbdd35c93b2f40f"}
Oct 30 22:29:57.077834 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:29:57.077611    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bfe0c7dbd084becf99d59962a0519e671526905b3097b29e5fbdd35c93b2f40f"
Oct 30 22:30:00.155943 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.155844    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"]
Oct 30 22:30:00.156490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.156358    5586 cpu_manager.go:414] "containerMap: removing container" podUID="fed66b60-0f21-4433-b567-449b3c05043d" containerName="revoke-keys"
Oct 30 22:30:00.156490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.156375    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fed66b60-0f21-4433-b567-449b3c05043d" containerName="revoke-keys"
Oct 30 22:30:00.156490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.156463    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="fed66b60-0f21-4433-b567-449b3c05043d" containerName="revoke-keys"
Oct 30 22:30:00.159997 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.159966    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"
Oct 30 22:30:00.213137 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.213105    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:30:00.215682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.215658    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:30:00.261024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.260978    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l2hv\" (UniqueName: \"kubernetes.io/projected/d98d8fb0-94f1-4b89-9146-e47654b6f4a7-kube-api-access-8l2hv\") pod \"maas-api-key-cleanup-29889990-dsqz9\" (UID: \"d98d8fb0-94f1-4b89-9146-e47654b6f4a7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"
Oct 30 22:30:00.261259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.261143    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d98d8fb0-94f1-4b89-9146-e47654b6f4a7-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889990-dsqz9\" (UID: \"d98d8fb0-94f1-4b89-9146-e47654b6f4a7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"
Oct 30 22:30:00.265113 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.265086    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:30:00.267783 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.267760    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"]
Oct 30 22:30:00.362382 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.362340    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l2hv\" (UniqueName: \"kubernetes.io/projected/d98d8fb0-94f1-4b89-9146-e47654b6f4a7-kube-api-access-8l2hv\") pod \"maas-api-key-cleanup-29889990-dsqz9\" (UID: \"d98d8fb0-94f1-4b89-9146-e47654b6f4a7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"
Oct 30 22:30:00.362567 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.362450    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d98d8fb0-94f1-4b89-9146-e47654b6f4a7-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889990-dsqz9\" (UID: \"d98d8fb0-94f1-4b89-9146-e47654b6f4a7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"
Oct 30 22:30:00.363112 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.363090    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d98d8fb0-94f1-4b89-9146-e47654b6f4a7-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889990-dsqz9\" (UID: \"d98d8fb0-94f1-4b89-9146-e47654b6f4a7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"
Oct 30 22:30:00.421494 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.421422    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l2hv\" (UniqueName: \"kubernetes.io/projected/d98d8fb0-94f1-4b89-9146-e47654b6f4a7-kube-api-access-8l2hv\") pod \"maas-api-key-cleanup-29889990-dsqz9\" (UID: \"d98d8fb0-94f1-4b89-9146-e47654b6f4a7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"
Oct 30 22:30:00.476153 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.476118    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"
Oct 30 22:30:00.856067 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:00.856041    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"]
Oct 30 22:30:00.858733 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:30:00.858688    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd98d8fb0_94f1_4b89_9146_e47654b6f4a7.slice/crio-c59b0e51424ce9ba944e2d597525a1c89bd56ebea82c2fbecd64d915006f3b97 WatchSource:0}: Error finding container c59b0e51424ce9ba944e2d597525a1c89bd56ebea82c2fbecd64d915006f3b97: Status 404 returned error can't find the container with id c59b0e51424ce9ba944e2d597525a1c89bd56ebea82c2fbecd64d915006f3b97
Oct 30 22:30:01.103769 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:01.103728    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9" event={"ID":"d98d8fb0-94f1-4b89-9146-e47654b6f4a7","Type":"ContainerStarted","Data":"c59b0e51424ce9ba944e2d597525a1c89bd56ebea82c2fbecd64d915006f3b97"}
Oct 30 22:30:02.111681 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:02.111644    5586 generic.go:356] "Generic (PLEG): container finished" podID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7" containerID="bc6df3cbc189e02a668de72b0162eaf717fc6bad99727c05bdf8e9cf3a133711" exitCode=60
Oct 30 22:30:02.112209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:02.111691    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9" event={"ID":"d98d8fb0-94f1-4b89-9146-e47654b6f4a7","Type":"ContainerDied","Data":"bc6df3cbc189e02a668de72b0162eaf717fc6bad99727c05bdf8e9cf3a133711"}
Oct 30 22:30:02.112209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:02.112029    5586 scope.go:122] "RemoveContainer" containerID="bc6df3cbc189e02a668de72b0162eaf717fc6bad99727c05bdf8e9cf3a133711"
Oct 30 22:30:03.119824 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:03.119790    5586 generic.go:356] "Generic (PLEG): container finished" podID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7" containerID="9f77abd5d65318565dbfc8cce8abd609165257de9e1846e71cbc74511f358ba6" exitCode=60
Oct 30 22:30:03.120365 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:03.119865    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9" event={"ID":"d98d8fb0-94f1-4b89-9146-e47654b6f4a7","Type":"ContainerDied","Data":"9f77abd5d65318565dbfc8cce8abd609165257de9e1846e71cbc74511f358ba6"}
Oct 30 22:30:03.120365 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:03.119925    5586 scope.go:122] "RemoveContainer" containerID="bc6df3cbc189e02a668de72b0162eaf717fc6bad99727c05bdf8e9cf3a133711"
Oct 30 22:30:03.120365 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:03.120208    5586 scope.go:122] "RemoveContainer" containerID="9f77abd5d65318565dbfc8cce8abd609165257de9e1846e71cbc74511f358ba6"
Oct 30 22:30:03.120559 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:30:03.120532    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889990-dsqz9_odh-ai-gateway-infra(d98d8fb0-94f1-4b89-9146-e47654b6f4a7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9" podUID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7"
Oct 30 22:30:04.128547 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:04.128513    5586 scope.go:122] "RemoveContainer" containerID="9f77abd5d65318565dbfc8cce8abd609165257de9e1846e71cbc74511f358ba6"
Oct 30 22:30:04.128969 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:30:04.128806    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889990-dsqz9_odh-ai-gateway-infra(d98d8fb0-94f1-4b89-9146-e47654b6f4a7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9" podUID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7"
Oct 30 22:30:08.418064 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:08.418026    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:08.926218 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:08.926169    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"]
Oct 30 22:30:08.931329 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:08.931302    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:08.949660 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:08.949623    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-podinfo\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:08.949660 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:08.949660    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-token\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:08.949899 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:08.949687    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-credential-socket\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:08.949899 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:08.949807    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-envoy\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:08.949899 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:08.949877    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68b70377-92ae-4856-b850-9d8a817f3ccc-istiod-ca-cert\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:08.950057 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:08.949909    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-data\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:08.950057 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:08.949962    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfxlt\" (UniqueName: \"kubernetes.io/projected/68b70377-92ae-4856-b850-9d8a817f3ccc-kube-api-access-mfxlt\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:08.950057 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:08.950034    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-workload-certs\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:08.950181 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:08.950075    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-workload-socket\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:08.990279 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:08.985094    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-5b7354-openshift-default-dockercfg-8ddvp\""
Oct 30 22:30:09.037196 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.037163    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"]
Oct 30 22:30:09.050757 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.050718    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-podinfo\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.050965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.050759    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-token\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.050965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.050797    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-credential-socket\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.050965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.050823    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-envoy\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.050965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.050846    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68b70377-92ae-4856-b850-9d8a817f3ccc-istiod-ca-cert\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.050965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.050872    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-data\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.050965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.050897    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfxlt\" (UniqueName: \"kubernetes.io/projected/68b70377-92ae-4856-b850-9d8a817f3ccc-kube-api-access-mfxlt\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.050965 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.050952    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-workload-certs\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.051428 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.051015    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-workload-socket\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.051495 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.051448    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-workload-certs\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.051495 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.051457    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-workload-socket\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.051662 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.051631    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-data\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.051747 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.051687    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-credential-socket\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.051811 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.051747    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68b70377-92ae-4856-b850-9d8a817f3ccc-istiod-ca-cert\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.053821 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.053786    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-envoy\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.053932 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.053886    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-podinfo\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.110675 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.110638    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-token\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.110882 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.110828    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfxlt\" (UniqueName: \"kubernetes.io/projected/68b70377-92ae-4856-b850-9d8a817f3ccc-kube-api-access-mfxlt\") pod \"e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") " pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.115132 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.115095    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:09.244454 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.244359    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:09.338547 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.338509    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:09.678887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.678857    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"]
Oct 30 22:30:09.680948 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:30:09.680916    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68b70377_92ae_4856_b850_9d8a817f3ccc.slice/crio-dfff112a4c29dc8da4d2287a5f46002e4519bff7b9c0cbcd06d23946b8d2fbf6 WatchSource:0}: Error finding container dfff112a4c29dc8da4d2287a5f46002e4519bff7b9c0cbcd06d23946b8d2fbf6: Status 404 returned error can't find the container with id dfff112a4c29dc8da4d2287a5f46002e4519bff7b9c0cbcd06d23946b8d2fbf6
Oct 30 22:30:09.683156 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.683121    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:09.683288 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.683202    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:09.683288 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:09.683265    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:10.047660 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:10.047554    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"]
Oct 30 22:30:10.050751 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:10.050720    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-kvftd"]
Oct 30 22:30:10.167099 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:10.167060    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f" event={"ID":"68b70377-92ae-4856-b850-9d8a817f3ccc","Type":"ContainerStarted","Data":"1a5668084365f9ab859a48824ca33ee495663624572f72d3f2cdcff54ab6d8fa"}
Oct 30 22:30:10.167346 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:10.167122    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f" event={"ID":"68b70377-92ae-4856-b850-9d8a817f3ccc","Type":"ContainerStarted","Data":"dfff112a4c29dc8da4d2287a5f46002e4519bff7b9c0cbcd06d23946b8d2fbf6"}
Oct 30 22:30:10.244817 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:10.244773    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:10.250503 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:10.250470    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:10.287083 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:10.287027    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f" podStartSLOduration=2.287011618 podStartE2EDuration="2.287011618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:10.284041642 +0000 UTC m=+3352.955945578" watchObservedRunningTime="2026-10-30 22:30:10.287011618 +0000 UTC m=+3352.958915558"
Oct 30 22:30:11.173642 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:11.173610    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:11.174641 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:11.174617    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:11.447715 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:11.447617    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4" path="/var/lib/kubelet/pods/2c1b51ea-d80f-4aaa-80e2-564e6aa7eff4/volumes"
Oct 30 22:30:13.266189 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:13.266148    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:13.433020 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:13.432982    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:13.602128 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:13.602046    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:14.422901 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:14.422858    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:15.440768 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:15.440739    5586 scope.go:122] "RemoveContainer" containerID="9f77abd5d65318565dbfc8cce8abd609165257de9e1846e71cbc74511f358ba6"
Oct 30 22:30:16.205093 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:16.205061    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9" event={"ID":"d98d8fb0-94f1-4b89-9146-e47654b6f4a7","Type":"ContainerStarted","Data":"986f3415f061db3bcd2399553bbe292ffae6f5d73d397909ef93aecbc66ccd5b"}
Oct 30 22:30:16.332391 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:16.332334    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9" podStartSLOduration=16.332317454 podStartE2EDuration="16.332317454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:16.329808506 +0000 UTC m=+3359.001712441" watchObservedRunningTime="2026-10-30 22:30:16.332317454 +0000 UTC m=+3359.004221389"
Oct 30 22:30:16.574247 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:16.574144    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:16.758128 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:16.758091    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:17.214217 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:17.214174    5586 generic.go:356] "Generic (PLEG): container finished" podID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7" containerID="986f3415f061db3bcd2399553bbe292ffae6f5d73d397909ef93aecbc66ccd5b" exitCode=60
Oct 30 22:30:17.214436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:17.214251    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9" event={"ID":"d98d8fb0-94f1-4b89-9146-e47654b6f4a7","Type":"ContainerDied","Data":"986f3415f061db3bcd2399553bbe292ffae6f5d73d397909ef93aecbc66ccd5b"}
Oct 30 22:30:17.214436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:17.214343    5586 scope.go:122] "RemoveContainer" containerID="9f77abd5d65318565dbfc8cce8abd609165257de9e1846e71cbc74511f358ba6"
Oct 30 22:30:17.214698 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:17.214677    5586 scope.go:122] "RemoveContainer" containerID="986f3415f061db3bcd2399553bbe292ffae6f5d73d397909ef93aecbc66ccd5b"
Oct 30 22:30:17.215257 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:30:17.214961    5586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889990-dsqz9_odh-ai-gateway-infra(d98d8fb0-94f1-4b89-9146-e47654b6f4a7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9" podUID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7"
Oct 30 22:30:17.344750 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:17.344719    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"]
Oct 30 22:30:18.361440 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:30:18.361405    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:30:18.363156 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:18.363132    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"
Oct 30 22:30:18.444293 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:18.444263    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d98d8fb0-94f1-4b89-9146-e47654b6f4a7-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d98d8fb0-94f1-4b89-9146-e47654b6f4a7-maas-api-service-ca\") pod \"d98d8fb0-94f1-4b89-9146-e47654b6f4a7\" (UID: \"d98d8fb0-94f1-4b89-9146-e47654b6f4a7\") "
Oct 30 22:30:18.444504 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:18.444345    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d98d8fb0-94f1-4b89-9146-e47654b6f4a7-kube-api-access-8l2hv\" (UniqueName: \"kubernetes.io/projected/d98d8fb0-94f1-4b89-9146-e47654b6f4a7-kube-api-access-8l2hv\") pod \"d98d8fb0-94f1-4b89-9146-e47654b6f4a7\" (UID: \"d98d8fb0-94f1-4b89-9146-e47654b6f4a7\") "
Oct 30 22:30:18.444684 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:18.444660    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d98d8fb0-94f1-4b89-9146-e47654b6f4a7-maas-api-service-ca" pod "d98d8fb0-94f1-4b89-9146-e47654b6f4a7" (UID: "d98d8fb0-94f1-4b89-9146-e47654b6f4a7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:18.446427 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:18.446402    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d98d8fb0-94f1-4b89-9146-e47654b6f4a7-kube-api-access-8l2hv" pod "d98d8fb0-94f1-4b89-9146-e47654b6f4a7" (UID: "d98d8fb0-94f1-4b89-9146-e47654b6f4a7"). InnerVolumeSpecName "kube-api-access-8l2hv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:18.545414 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:18.545338    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d98d8fb0-94f1-4b89-9146-e47654b6f4a7-maas-api-service-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:18.545414 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:18.545364    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l2hv\" (UniqueName: \"kubernetes.io/projected/d98d8fb0-94f1-4b89-9146-e47654b6f4a7-kube-api-access-8l2hv\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:19.229842 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:19.229807    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"
Oct 30 22:30:19.229842 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:19.229834    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9" event={"ID":"d98d8fb0-94f1-4b89-9146-e47654b6f4a7","Type":"ContainerDied","Data":"c59b0e51424ce9ba944e2d597525a1c89bd56ebea82c2fbecd64d915006f3b97"}
Oct 30 22:30:19.230281 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:19.229894    5586 scope.go:122] "RemoveContainer" containerID="986f3415f061db3bcd2399553bbe292ffae6f5d73d397909ef93aecbc66ccd5b"
Oct 30 22:30:19.394930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:19.394889    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"]
Oct 30 22:30:19.446188 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:19.446157    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889990-dsqz9"]
Oct 30 22:30:19.832333 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:19.832301    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:20.020193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:20.020155    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:21.060755 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:21.060714    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"]
Oct 30 22:30:21.108530 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:21.108492    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-6fz97"]
Oct 30 22:30:21.389945 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:21.389856    5586 scope.go:122] "RemoveContainer" containerID="0dbb7cc8ab0d0e6073d63023a73c787a10828da59692676f2200dd123beb12d0"
Oct 30 22:30:21.399292 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:21.399270    5586 scope.go:122] "RemoveContainer" containerID="814c6d406f4ab8e12b465a255fd27e971b4d31985a50891a90c4c9966a223a84"
Oct 30 22:30:21.408309 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:21.408288    5586 scope.go:122] "RemoveContainer" containerID="5f01beabeb623de8089d685a12e0b91521eda0cb3e5ec09cd3ff88a0471dd5bf"
Oct 30 22:30:21.447568 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:21.447529    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20a60a01-46ac-45b4-8530-24621cbcea7a" path="/var/lib/kubelet/pods/20a60a01-46ac-45b4-8530-24621cbcea7a/volumes"
Oct 30 22:30:21.448112 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:21.448085    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7" path="/var/lib/kubelet/pods/d98d8fb0-94f1-4b89-9146-e47654b6f4a7/volumes"
Oct 30 22:30:24.929279 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:24.929221    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:25.299745 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:25.299641    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:29.932870 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:29.932836    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:34.791551 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:34.791499    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:35.085837 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:35.085749    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:37.342272 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:37.342217    5586 generic.go:356] "Generic (PLEG): container finished" podID="68b70377-92ae-4856-b850-9d8a817f3ccc" containerID="1a5668084365f9ab859a48824ca33ee495663624572f72d3f2cdcff54ab6d8fa" exitCode=0
Oct 30 22:30:37.342700 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:37.342295    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f" event={"ID":"68b70377-92ae-4856-b850-9d8a817f3ccc","Type":"ContainerDied","Data":"1a5668084365f9ab859a48824ca33ee495663624572f72d3f2cdcff54ab6d8fa"}
Oct 30 22:30:37.342700 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:37.342679    5586 scope.go:122] "RemoveContainer" containerID="1a5668084365f9ab859a48824ca33ee495663624572f72d3f2cdcff54ab6d8fa"
Oct 30 22:30:37.343813 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:37.343790    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:37.343876 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:37.343853    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:37.343917 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:37.343885    5586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 22:30:38.196815 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:38.196778    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:38.353814 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:38.353776    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f" event={"ID":"68b70377-92ae-4856-b850-9d8a817f3ccc","Type":"ContainerStarted","Data":"d1909c8d41b6f1469e709f52b0cb8456bc7efdbdaa500e7079b416615dc4aa05"}
Oct 30 22:30:38.903166 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:38.903120    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:39.211923 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:39.211879    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:39.246459 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:39.246414    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:39.247817 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:39.247786    5586 patch_prober.go:29] interesting pod/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.179:15021/healthz/ready\": dial tcp 10.134.0.179:15021: connect: connection refused" start-of-body=
Oct 30 22:30:39.247950 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:39.247850    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f" podUID="68b70377-92ae-4856-b850-9d8a817f3ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.179:15021/healthz/ready\": dial tcp 10.134.0.179:15021: connect: connection refused"
Oct 30 22:30:39.825262 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:39.821548    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:40.246772 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:40.246731    5586 patch_prober.go:29] interesting pod/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.179:15021/healthz/ready\": dial tcp 10.134.0.179:15021: connect: connection refused" start-of-body=
Oct 30 22:30:40.246972 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:40.246801    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f" podUID="68b70377-92ae-4856-b850-9d8a817f3ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.179:15021/healthz/ready\": dial tcp 10.134.0.179:15021: connect: connection refused"
Oct 30 22:30:41.174009 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:41.173975    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:41.246587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:41.246550    5586 patch_prober.go:29] interesting pod/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.179:15021/healthz/ready\": dial tcp 10.134.0.179:15021: connect: connection refused" start-of-body=
Oct 30 22:30:41.246815 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:41.246605    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f" podUID="68b70377-92ae-4856-b850-9d8a817f3ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.179:15021/healthz/ready\": dial tcp 10.134.0.179:15021: connect: connection refused"
Oct 30 22:30:42.246691 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.246649    5586 patch_prober.go:29] interesting pod/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.179:15021/healthz/ready\": dial tcp 10.134.0.179:15021: connect: connection refused" start-of-body=
Oct 30 22:30:42.247226 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.246750    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f" podUID="68b70377-92ae-4856-b850-9d8a817f3ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.179:15021/healthz/ready\": dial tcp 10.134.0.179:15021: connect: connection refused"
Oct 30 22:30:42.593610 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.593520    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"]
Oct 30 22:30:42.594049 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.594028    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7" containerName="cleanup"
Oct 30 22:30:42.594151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.594051    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7" containerName="cleanup"
Oct 30 22:30:42.594151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.594067    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7" containerName="cleanup"
Oct 30 22:30:42.594151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.594074    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7" containerName="cleanup"
Oct 30 22:30:42.594345 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.594246    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7" containerName="cleanup"
Oct 30 22:30:42.594345 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.594261    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7" containerName="cleanup"
Oct 30 22:30:42.594345 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.594273    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7" containerName="cleanup"
Oct 30 22:30:42.597739 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.597718    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.675025 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.674992    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-5b7354\"/\"kube-root-ca.crt\""
Oct 30 22:30:42.675025 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.675022    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-5b7354\"/\"openshift-service-ca.crt\""
Oct 30 22:30:42.699049 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.699014    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-5b7354\"/\"default-dockercfg-zgt6p\""
Oct 30 22:30:42.699229 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.699090    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-5b7354\"/\"simulator-single-main-5b7354-kserve-self-signed-certs\""
Oct 30 22:30:42.701475 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.701455    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"]
Oct 30 22:30:42.709642 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.709623    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-tmp-dir\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.709713 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.709651    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz4t7\" (UniqueName: \"kubernetes.io/projected/51759e2d-609a-4696-b9e2-fe934df66643-kube-api-access-mz4t7\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.709752 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.709726    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51759e2d-609a-4696-b9e2-fe934df66643-tls-certs\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.709883 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.709867    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-model-cache\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.709937 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.709924    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-dshm\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.709978 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.709948    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-home\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.756631 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.756593    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:42.811508 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.811470    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-model-cache\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.811737 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.811523    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-dshm\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.811737 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.811541    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-home\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.811737 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.811566    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-tmp-dir\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.811737 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.811585    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz4t7\" (UniqueName: \"kubernetes.io/projected/51759e2d-609a-4696-b9e2-fe934df66643-kube-api-access-mz4t7\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.811737 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.811616    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51759e2d-609a-4696-b9e2-fe934df66643-tls-certs\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.812058 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.812018    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-model-cache\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.812115 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.812060    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-tmp-dir\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.812115 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.812066    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-home\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.813926 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.813902    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-dshm\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.814145 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.814127    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51759e2d-609a-4696-b9e2-fe934df66643-tls-certs\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.868632 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.868548    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz4t7\" (UniqueName: \"kubernetes.io/projected/51759e2d-609a-4696-b9e2-fe934df66643-kube-api-access-mz4t7\") pod \"simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") " pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:42.910285 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:42.910208    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:43.244942 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:43.244889    5586 patch_prober.go:29] interesting pod/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.179:15021/healthz/ready\": dial tcp 10.134.0.179:15021: connect: connection refused" start-of-body=
Oct 30 22:30:43.245140 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:43.244977    5586 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f" podUID="68b70377-92ae-4856-b850-9d8a817f3ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.179:15021/healthz/ready\": dial tcp 10.134.0.179:15021: connect: connection refused"
Oct 30 22:30:43.314201 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:43.314173    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"]
Oct 30 22:30:43.317013 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:30:43.316976    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51759e2d_609a_4696_b9e2_fe934df66643.slice/crio-5cb2bfabd5398b173333577ec0929a14834b01e59a8bf3482da49f421380fc32 WatchSource:0}: Error finding container 5cb2bfabd5398b173333577ec0929a14834b01e59a8bf3482da49f421380fc32: Status 404 returned error can't find the container with id 5cb2bfabd5398b173333577ec0929a14834b01e59a8bf3482da49f421380fc32
Oct 30 22:30:43.387061 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:43.387026    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8" event={"ID":"51759e2d-609a-4696-b9e2-fe934df66643","Type":"ContainerStarted","Data":"bc4455b6f4e167e75b6c5291c80ed336568529dfff789352e3b14f2fef28597e"}
Oct 30 22:30:43.387292 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:43.387072    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8" event={"ID":"51759e2d-609a-4696-b9e2-fe934df66643","Type":"ContainerStarted","Data":"5cb2bfabd5398b173333577ec0929a14834b01e59a8bf3482da49f421380fc32"}
Oct 30 22:30:43.506676 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:43.506556    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8" podStartSLOduration=1.5065423519999999 podStartE2EDuration="1.506542352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:43.504505903 +0000 UTC m=+3386.176409839" watchObservedRunningTime="2026-10-30 22:30:43.506542352 +0000 UTC m=+3386.178446286"
Oct 30 22:30:43.910797 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:43.910756    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:44.249939 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:44.249856    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:44.251002 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:44.250982    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:30:44.497575 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:44.497538    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:48.361129 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:30:48.361093    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:30:49.295975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:49.295934    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:52.910913 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:52.910874    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:52.923802 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:52.923772    5586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:53.457867 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:53.457836    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:30:54.009195 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:54.009156    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:55.048220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:55.048181    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"]
Oct 30 22:30:55.097504 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:55.097474    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-748mh"]
Oct 30 22:30:55.447327 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:55.447281    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed2517b8-7ae0-43b0-9724-15a73df80e70" path="/var/lib/kubelet/pods/ed2517b8-7ae0-43b0-9724-15a73df80e70/volumes"
Oct 30 22:30:58.889343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:30:58.889304    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:03.679636 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:03.679582    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:08.659995 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:08.659951    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:13.577166 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:13.577124    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:18.361270 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:31:18.361210    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:31:18.559060 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:18.559011    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:21.452261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:21.452216    5586 scope.go:122] "RemoveContainer" containerID="ff5301b389db65116b8e8b372b561270c3fd139308db33a382cb40f84a868a7d"
Oct 30 22:31:21.461732 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:21.461707    5586 scope.go:122] "RemoveContainer" containerID="910e2a63b3e85b2b6767358fe936408a4aed5a11556f009c5c8fdf41ef237426"
Oct 30 22:31:21.767630 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:21.767516    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal" podUID="6ea3e08f52bca920602c76ad35bcaf69" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Oct 30 22:31:23.339173 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:23.339122    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:23.590469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:23.590365    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:24.724567 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:24.724527    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:26.442097 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.442047    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"]
Oct 30 22:31:26.442535 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.442505    5586 cpu_manager.go:414] "containerMap: removing container" podUID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7" containerName="cleanup"
Oct 30 22:31:26.442535 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.442516    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d98d8fb0-94f1-4b89-9146-e47654b6f4a7" containerName="cleanup"
Oct 30 22:31:26.447186 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.447166    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"
Oct 30 22:31:26.503544 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.503511    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"]
Oct 30 22:31:26.539758 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.539712    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df0ab2a5-f98a-4375-8b8c-3744fc903726-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-573222d1-4nvmg\" (UID: \"df0ab2a5-f98a-4375-8b8c-3744fc903726\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"
Oct 30 22:31:26.539947 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.539885    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df0ab2a5-f98a-4375-8b8c-3744fc903726-maas-api-xks-ca\") pod \"maas-api-revoke-sub-573222d1-4nvmg\" (UID: \"df0ab2a5-f98a-4375-8b8c-3744fc903726\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"
Oct 30 22:31:26.540020 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.540000    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k89zl\" (UniqueName: \"kubernetes.io/projected/df0ab2a5-f98a-4375-8b8c-3744fc903726-kube-api-access-k89zl\") pod \"maas-api-revoke-sub-573222d1-4nvmg\" (UID: \"df0ab2a5-f98a-4375-8b8c-3744fc903726\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"
Oct 30 22:31:26.549602 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.549566    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:31:26.549602 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.549584    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:31:26.549602 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.549588    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:31:26.549602 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.549570    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:31:26.641061 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.641030    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df0ab2a5-f98a-4375-8b8c-3744fc903726-maas-api-xks-ca\") pod \"maas-api-revoke-sub-573222d1-4nvmg\" (UID: \"df0ab2a5-f98a-4375-8b8c-3744fc903726\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"
Oct 30 22:31:26.641252 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.641099    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k89zl\" (UniqueName: \"kubernetes.io/projected/df0ab2a5-f98a-4375-8b8c-3744fc903726-kube-api-access-k89zl\") pod \"maas-api-revoke-sub-573222d1-4nvmg\" (UID: \"df0ab2a5-f98a-4375-8b8c-3744fc903726\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"
Oct 30 22:31:26.641252 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.641160    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df0ab2a5-f98a-4375-8b8c-3744fc903726-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-573222d1-4nvmg\" (UID: \"df0ab2a5-f98a-4375-8b8c-3744fc903726\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"
Oct 30 22:31:26.641818 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.641798    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df0ab2a5-f98a-4375-8b8c-3744fc903726-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-573222d1-4nvmg\" (UID: \"df0ab2a5-f98a-4375-8b8c-3744fc903726\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"
Oct 30 22:31:26.643377 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.643353    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df0ab2a5-f98a-4375-8b8c-3744fc903726-maas-api-xks-ca\") pod \"maas-api-revoke-sub-573222d1-4nvmg\" (UID: \"df0ab2a5-f98a-4375-8b8c-3744fc903726\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"
Oct 30 22:31:26.702581 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.702506    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k89zl\" (UniqueName: \"kubernetes.io/projected/df0ab2a5-f98a-4375-8b8c-3744fc903726-kube-api-access-k89zl\") pod \"maas-api-revoke-sub-573222d1-4nvmg\" (UID: \"df0ab2a5-f98a-4375-8b8c-3744fc903726\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"
Oct 30 22:31:26.760498 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:26.760459    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"
Oct 30 22:31:27.143491 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:27.143463    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"]
Oct 30 22:31:27.145918 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:31:27.145887    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf0ab2a5_f98a_4375_8b8c_3744fc903726.slice/crio-d136be6141213c2f32b26a5a93e34e6c08f4fffc5997462f4600bd28b90ddd48 WatchSource:0}: Error finding container d136be6141213c2f32b26a5a93e34e6c08f4fffc5997462f4600bd28b90ddd48: Status 404 returned error can't find the container with id d136be6141213c2f32b26a5a93e34e6c08f4fffc5997462f4600bd28b90ddd48
Oct 30 22:31:27.663839 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:27.663799    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg" event={"ID":"df0ab2a5-f98a-4375-8b8c-3744fc903726","Type":"ContainerStarted","Data":"7396d72e8eba49193c261915f6bd7df3da1fd17c0207397f77e4c93fae5f1c25"}
Oct 30 22:31:27.664284 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:27.663862    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg" event={"ID":"df0ab2a5-f98a-4375-8b8c-3744fc903726","Type":"ContainerStarted","Data":"d136be6141213c2f32b26a5a93e34e6c08f4fffc5997462f4600bd28b90ddd48"}
Oct 30 22:31:27.791402 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:27.791297    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg" podStartSLOduration=1.791283011 podStartE2EDuration="1.791283011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:31:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:31:27.788637345 +0000 UTC m=+3430.460541280" watchObservedRunningTime="2026-10-30 22:31:27.791283011 +0000 UTC m=+3430.463186945"
Oct 30 22:31:28.613912 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:28.613788    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:28.672390 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:28.672352    5586 generic.go:356] "Generic (PLEG): container finished" podID="df0ab2a5-f98a-4375-8b8c-3744fc903726" containerID="7396d72e8eba49193c261915f6bd7df3da1fd17c0207397f77e4c93fae5f1c25" exitCode=0
Oct 30 22:31:28.672823 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:28.672422    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg" event={"ID":"df0ab2a5-f98a-4375-8b8c-3744fc903726","Type":"ContainerDied","Data":"7396d72e8eba49193c261915f6bd7df3da1fd17c0207397f77e4c93fae5f1c25"}
Oct 30 22:31:29.830858 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:29.830829    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"
Oct 30 22:31:29.974910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:29.974812    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df0ab2a5-f98a-4375-8b8c-3744fc903726-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df0ab2a5-f98a-4375-8b8c-3744fc903726-maas-api-openshift-ca\") pod \"df0ab2a5-f98a-4375-8b8c-3744fc903726\" (UID: \"df0ab2a5-f98a-4375-8b8c-3744fc903726\") "
Oct 30 22:31:29.974910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:29.974886    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df0ab2a5-f98a-4375-8b8c-3744fc903726-kube-api-access-k89zl\" (UniqueName: \"kubernetes.io/projected/df0ab2a5-f98a-4375-8b8c-3744fc903726-kube-api-access-k89zl\") pod \"df0ab2a5-f98a-4375-8b8c-3744fc903726\" (UID: \"df0ab2a5-f98a-4375-8b8c-3744fc903726\") "
Oct 30 22:31:29.975161 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:29.974930    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df0ab2a5-f98a-4375-8b8c-3744fc903726-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df0ab2a5-f98a-4375-8b8c-3744fc903726-maas-api-xks-ca\") pod \"df0ab2a5-f98a-4375-8b8c-3744fc903726\" (UID: \"df0ab2a5-f98a-4375-8b8c-3744fc903726\") "
Oct 30 22:31:29.975298 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:29.975269    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df0ab2a5-f98a-4375-8b8c-3744fc903726-maas-api-openshift-ca" pod "df0ab2a5-f98a-4375-8b8c-3744fc903726" (UID: "df0ab2a5-f98a-4375-8b8c-3744fc903726"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:31:29.977144 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:29.977112    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df0ab2a5-f98a-4375-8b8c-3744fc903726-kube-api-access-k89zl" pod "df0ab2a5-f98a-4375-8b8c-3744fc903726" (UID: "df0ab2a5-f98a-4375-8b8c-3744fc903726"). InnerVolumeSpecName "kube-api-access-k89zl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:29.977306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:29.977222    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df0ab2a5-f98a-4375-8b8c-3744fc903726-maas-api-xks-ca" pod "df0ab2a5-f98a-4375-8b8c-3744fc903726" (UID: "df0ab2a5-f98a-4375-8b8c-3744fc903726"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:31:30.076072 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:30.076031    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/df0ab2a5-f98a-4375-8b8c-3744fc903726-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:30.076072 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:30.076062    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k89zl\" (UniqueName: \"kubernetes.io/projected/df0ab2a5-f98a-4375-8b8c-3744fc903726-kube-api-access-k89zl\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:30.076072 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:30.076075    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/df0ab2a5-f98a-4375-8b8c-3744fc903726-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:30.685638 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:30.685608    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"
Oct 30 22:31:30.685832 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:30.685637    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg" event={"ID":"df0ab2a5-f98a-4375-8b8c-3744fc903726","Type":"ContainerDied","Data":"d136be6141213c2f32b26a5a93e34e6c08f4fffc5997462f4600bd28b90ddd48"}
Oct 30 22:31:30.685832 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:30.685685    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d136be6141213c2f32b26a5a93e34e6c08f4fffc5997462f4600bd28b90ddd48"
Oct 30 22:31:33.078990 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:33.078946    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"]
Oct 30 22:31:33.127499 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:33.127450    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-vhsl4"]
Oct 30 22:31:33.446705 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:33.446665    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0db15b9e-2e58-432a-990b-fbe7946d9f33" path="/var/lib/kubelet/pods/0db15b9e-2e58-432a-990b-fbe7946d9f33/volumes"
Oct 30 22:31:33.604804 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:33.604760    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:35.352808 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:35.352770    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:38.200014 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.199972    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:38.578830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.578740    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:38.732538 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.732495    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"]
Oct 30 22:31:38.732938 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.732925    5586 cpu_manager.go:414] "containerMap: removing container" podUID="df0ab2a5-f98a-4375-8b8c-3744fc903726" containerName="revoke-keys"
Oct 30 22:31:38.732983 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.732939    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df0ab2a5-f98a-4375-8b8c-3744fc903726" containerName="revoke-keys"
Oct 30 22:31:38.733034 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.733025    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="df0ab2a5-f98a-4375-8b8c-3744fc903726" containerName="revoke-keys"
Oct 30 22:31:38.736506 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.736488    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"
Oct 30 22:31:38.792288 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.792226    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:31:38.841578 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.841470    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:31:38.841578 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.841487    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:31:38.841776 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.841650    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:31:38.844245 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.844210    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"]
Oct 30 22:31:38.872271 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.872212    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/871270f6-45aa-4e80-bcf2-13c77b76baae-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh\" (UID: \"871270f6-45aa-4e80-bcf2-13c77b76baae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"
Oct 30 22:31:38.872271 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.872267    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/871270f6-45aa-4e80-bcf2-13c77b76baae-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh\" (UID: \"871270f6-45aa-4e80-bcf2-13c77b76baae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"
Oct 30 22:31:38.872482 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.872427    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h8kr\" (UniqueName: \"kubernetes.io/projected/871270f6-45aa-4e80-bcf2-13c77b76baae-kube-api-access-5h8kr\") pod \"maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh\" (UID: \"871270f6-45aa-4e80-bcf2-13c77b76baae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"
Oct 30 22:31:38.974057 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.974013    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h8kr\" (UniqueName: \"kubernetes.io/projected/871270f6-45aa-4e80-bcf2-13c77b76baae-kube-api-access-5h8kr\") pod \"maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh\" (UID: \"871270f6-45aa-4e80-bcf2-13c77b76baae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"
Oct 30 22:31:38.974292 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.974077    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/871270f6-45aa-4e80-bcf2-13c77b76baae-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh\" (UID: \"871270f6-45aa-4e80-bcf2-13c77b76baae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"
Oct 30 22:31:38.974292 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.974103    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/871270f6-45aa-4e80-bcf2-13c77b76baae-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh\" (UID: \"871270f6-45aa-4e80-bcf2-13c77b76baae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"
Oct 30 22:31:38.974755 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.974735    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/871270f6-45aa-4e80-bcf2-13c77b76baae-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh\" (UID: \"871270f6-45aa-4e80-bcf2-13c77b76baae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"
Oct 30 22:31:38.976559 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:38.976539    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/871270f6-45aa-4e80-bcf2-13c77b76baae-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh\" (UID: \"871270f6-45aa-4e80-bcf2-13c77b76baae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"
Oct 30 22:31:39.031501 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:39.031458    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h8kr\" (UniqueName: \"kubernetes.io/projected/871270f6-45aa-4e80-bcf2-13c77b76baae-kube-api-access-5h8kr\") pod \"maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh\" (UID: \"871270f6-45aa-4e80-bcf2-13c77b76baae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"
Oct 30 22:31:39.049161 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:39.049128    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"
Oct 30 22:31:39.426438 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:39.426406    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"]
Oct 30 22:31:39.428900 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:31:39.428872    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod871270f6_45aa_4e80_bcf2_13c77b76baae.slice/crio-10edf886319a3c2fd26a68bdaf4e5f7746a3dbf97d979f0dc4a5055f7cc056f4 WatchSource:0}: Error finding container 10edf886319a3c2fd26a68bdaf4e5f7746a3dbf97d979f0dc4a5055f7cc056f4: Status 404 returned error can't find the container with id 10edf886319a3c2fd26a68bdaf4e5f7746a3dbf97d979f0dc4a5055f7cc056f4
Oct 30 22:31:39.739574 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:39.739539    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh" event={"ID":"871270f6-45aa-4e80-bcf2-13c77b76baae","Type":"ContainerStarted","Data":"10edf886319a3c2fd26a68bdaf4e5f7746a3dbf97d979f0dc4a5055f7cc056f4"}
Oct 30 22:31:40.746739 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:40.746694    5586 generic.go:356] "Generic (PLEG): container finished" podID="871270f6-45aa-4e80-bcf2-13c77b76baae" containerID="3eaa42d13c44676a757f7185e6ed0774a72f4815906738baefe225b69b8d98f5" exitCode=0
Oct 30 22:31:40.747148 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:40.746756    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh" event={"ID":"871270f6-45aa-4e80-bcf2-13c77b76baae","Type":"ContainerDied","Data":"3eaa42d13c44676a757f7185e6ed0774a72f4815906738baefe225b69b8d98f5"}
Oct 30 22:31:41.901393 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:41.901358    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"
Oct 30 22:31:42.006607 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:42.006506    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/871270f6-45aa-4e80-bcf2-13c77b76baae-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/871270f6-45aa-4e80-bcf2-13c77b76baae-maas-api-openshift-ca\") pod \"871270f6-45aa-4e80-bcf2-13c77b76baae\" (UID: \"871270f6-45aa-4e80-bcf2-13c77b76baae\") "
Oct 30 22:31:42.006770 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:42.006722    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/871270f6-45aa-4e80-bcf2-13c77b76baae-kube-api-access-5h8kr\" (UniqueName: \"kubernetes.io/projected/871270f6-45aa-4e80-bcf2-13c77b76baae-kube-api-access-5h8kr\") pod \"871270f6-45aa-4e80-bcf2-13c77b76baae\" (UID: \"871270f6-45aa-4e80-bcf2-13c77b76baae\") "
Oct 30 22:31:42.006835 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:42.006784    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/871270f6-45aa-4e80-bcf2-13c77b76baae-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/871270f6-45aa-4e80-bcf2-13c77b76baae-maas-api-xks-ca\") pod \"871270f6-45aa-4e80-bcf2-13c77b76baae\" (UID: \"871270f6-45aa-4e80-bcf2-13c77b76baae\") "
Oct 30 22:31:42.006893 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:42.006835    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/871270f6-45aa-4e80-bcf2-13c77b76baae-maas-api-openshift-ca" pod "871270f6-45aa-4e80-bcf2-13c77b76baae" (UID: "871270f6-45aa-4e80-bcf2-13c77b76baae"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:31:42.007117 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:42.007096    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/871270f6-45aa-4e80-bcf2-13c77b76baae-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:42.009042 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:42.009016    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/871270f6-45aa-4e80-bcf2-13c77b76baae-maas-api-xks-ca" pod "871270f6-45aa-4e80-bcf2-13c77b76baae" (UID: "871270f6-45aa-4e80-bcf2-13c77b76baae"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:31:42.009138 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:42.009023    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/871270f6-45aa-4e80-bcf2-13c77b76baae-kube-api-access-5h8kr" pod "871270f6-45aa-4e80-bcf2-13c77b76baae" (UID: "871270f6-45aa-4e80-bcf2-13c77b76baae"). InnerVolumeSpecName "kube-api-access-5h8kr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:42.107984 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:42.107944    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5h8kr\" (UniqueName: \"kubernetes.io/projected/871270f6-45aa-4e80-bcf2-13c77b76baae-kube-api-access-5h8kr\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:42.107984 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:42.107984    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/871270f6-45aa-4e80-bcf2-13c77b76baae-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:42.762441 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:42.762410    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"
Oct 30 22:31:42.762441 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:42.762423    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh" event={"ID":"871270f6-45aa-4e80-bcf2-13c77b76baae","Type":"ContainerDied","Data":"10edf886319a3c2fd26a68bdaf4e5f7746a3dbf97d979f0dc4a5055f7cc056f4"}
Oct 30 22:31:42.762667 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:42.762470    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10edf886319a3c2fd26a68bdaf4e5f7746a3dbf97d979f0dc4a5055f7cc056f4"
Oct 30 22:31:43.137044 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:43.136952    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"]
Oct 30 22:31:43.231699 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:43.231660    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:43.757735 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:43.757701    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:48.360823 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:31:48.360783    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:31:48.591139 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:48.591099    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:49.518677 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:49.518474    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:49.705220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:49.705180    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:53.267712 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:53.267674    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:57.450535 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:57.450493    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:58.163012 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.162972    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8" podUID="51759e2d-609a-4696-b9e2-fe934df66643" containerName="main" containerID="cri-o://bc4455b6f4e167e75b6c5291c80ed336568529dfff789352e3b14f2fef28597e" gracePeriod=45
Oct 30 22:31:58.423204 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.423137    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:31:58.479938 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.479897    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-dshm\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-dshm\") pod \"51759e2d-609a-4696-b9e2-fe934df66643\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") "
Oct 30 22:31:58.480480 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.479951    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-tmp-dir\") pod \"51759e2d-609a-4696-b9e2-fe934df66643\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") "
Oct 30 22:31:58.480480 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.479988    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51759e2d-609a-4696-b9e2-fe934df66643-tls-certs\" (UniqueName: \"kubernetes.io/secret/51759e2d-609a-4696-b9e2-fe934df66643-tls-certs\") pod \"51759e2d-609a-4696-b9e2-fe934df66643\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") "
Oct 30 22:31:58.480480 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.480018    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51759e2d-609a-4696-b9e2-fe934df66643-kube-api-access-mz4t7\" (UniqueName: \"kubernetes.io/projected/51759e2d-609a-4696-b9e2-fe934df66643-kube-api-access-mz4t7\") pod \"51759e2d-609a-4696-b9e2-fe934df66643\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") "
Oct 30 22:31:58.480480 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.480094    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-home\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-home\") pod \"51759e2d-609a-4696-b9e2-fe934df66643\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") "
Oct 30 22:31:58.480480 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.480141    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-model-cache\") pod \"51759e2d-609a-4696-b9e2-fe934df66643\" (UID: \"51759e2d-609a-4696-b9e2-fe934df66643\") "
Oct 30 22:31:58.480480 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.480273    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-tmp-dir" pod "51759e2d-609a-4696-b9e2-fe934df66643" (UID: "51759e2d-609a-4696-b9e2-fe934df66643"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:58.480480 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.480421    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-home" pod "51759e2d-609a-4696-b9e2-fe934df66643" (UID: "51759e2d-609a-4696-b9e2-fe934df66643"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:58.480806 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.480485    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-model-cache" pod "51759e2d-609a-4696-b9e2-fe934df66643" (UID: "51759e2d-609a-4696-b9e2-fe934df66643"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:58.480806 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.480713    5586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-home\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:58.480806 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.480735    5586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-model-cache\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:58.480806 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.480749    5586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-tmp-dir\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:58.482421 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.482387    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51759e2d-609a-4696-b9e2-fe934df66643-tls-certs" pod "51759e2d-609a-4696-b9e2-fe934df66643" (UID: "51759e2d-609a-4696-b9e2-fe934df66643"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:31:58.482540 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.482437    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-dshm" pod "51759e2d-609a-4696-b9e2-fe934df66643" (UID: "51759e2d-609a-4696-b9e2-fe934df66643"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:58.482540 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.482483    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51759e2d-609a-4696-b9e2-fe934df66643-kube-api-access-mz4t7" pod "51759e2d-609a-4696-b9e2-fe934df66643" (UID: "51759e2d-609a-4696-b9e2-fe934df66643"). InnerVolumeSpecName "kube-api-access-mz4t7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:58.582767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.582729    5586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51759e2d-609a-4696-b9e2-fe934df66643-dshm\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:58.582767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.582767    5586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51759e2d-609a-4696-b9e2-fe934df66643-tls-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:58.582983 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.582782    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz4t7\" (UniqueName: \"kubernetes.io/projected/51759e2d-609a-4696-b9e2-fe934df66643-kube-api-access-mz4t7\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:58.862092 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.861999    5586 generic.go:356] "Generic (PLEG): container finished" podID="51759e2d-609a-4696-b9e2-fe934df66643" containerID="bc4455b6f4e167e75b6c5291c80ed336568529dfff789352e3b14f2fef28597e" exitCode=0
Oct 30 22:31:58.862092 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.862049    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8" event={"ID":"51759e2d-609a-4696-b9e2-fe934df66643","Type":"ContainerDied","Data":"bc4455b6f4e167e75b6c5291c80ed336568529dfff789352e3b14f2fef28597e"}
Oct 30 22:31:58.862092 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.862089    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8" event={"ID":"51759e2d-609a-4696-b9e2-fe934df66643","Type":"ContainerDied","Data":"5cb2bfabd5398b173333577ec0929a14834b01e59a8bf3482da49f421380fc32"}
Oct 30 22:31:58.862428 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.862110    5586 scope.go:122] "RemoveContainer" containerID="bc4455b6f4e167e75b6c5291c80ed336568529dfff789352e3b14f2fef28597e"
Oct 30 22:31:58.862428 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.862056    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"
Oct 30 22:31:58.872428 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.872408    5586 scope.go:122] "RemoveContainer" containerID="bc4455b6f4e167e75b6c5291c80ed336568529dfff789352e3b14f2fef28597e"
Oct 30 22:31:58.872715 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:31:58.872693    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc4455b6f4e167e75b6c5291c80ed336568529dfff789352e3b14f2fef28597e\": container with ID starting with bc4455b6f4e167e75b6c5291c80ed336568529dfff789352e3b14f2fef28597e not found: ID does not exist" containerID="bc4455b6f4e167e75b6c5291c80ed336568529dfff789352e3b14f2fef28597e"
Oct 30 22:31:58.872787 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.872727    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc4455b6f4e167e75b6c5291c80ed336568529dfff789352e3b14f2fef28597e"} err="failed to get container status \"bc4455b6f4e167e75b6c5291c80ed336568529dfff789352e3b14f2fef28597e\": rpc error: code = NotFound desc = could not find container \"bc4455b6f4e167e75b6c5291c80ed336568529dfff789352e3b14f2fef28597e\": container with ID starting with bc4455b6f4e167e75b6c5291c80ed336568529dfff789352e3b14f2fef28597e not found: ID does not exist"
Oct 30 22:31:58.972519 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:58.972448    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:59.024952 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:59.024913    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"]
Oct 30 22:31:59.075621 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:59.075586    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-5b7354/simulator-single-main-5b7354-kserve-5b55f9fbd6-ltqd8"]
Oct 30 22:31:59.239575 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:59.239536    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:59.446332 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:59.446290    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51759e2d-609a-4696-b9e2-fe934df66643" path="/var/lib/kubelet/pods/51759e2d-609a-4696-b9e2-fe934df66643/volumes"
Oct 30 22:31:59.546409 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:59.546311    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"]
Oct 30 22:31:59.548450 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:59.548426    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-5b7354-csdsh"]
Oct 30 22:31:59.997268 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:31:59.997207    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:01.446330 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:01.446291    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="871270f6-45aa-4e80-bcf2-13c77b76baae" path="/var/lib/kubelet/pods/871270f6-45aa-4e80-bcf2-13c77b76baae/volumes"
Oct 30 22:32:06.094411 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:06.094377    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:06.398431 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:06.398327    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:06.440891 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:06.440862    5586 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f" secret="" err="secret \"e2e-worker-single-main-5b7354-openshift-default-dockercfg-8ddvp\" not found"
Oct 30 22:32:06.461092 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:06.461054    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"]
Oct 30 22:32:06.707616 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:06.707576    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:06.913245 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:06.913184    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f" podUID="68b70377-92ae-4856-b850-9d8a817f3ccc" containerName="istio-proxy" containerID="cri-o://d1909c8d41b6f1469e709f52b0cb8456bc7efdbdaa500e7079b416615dc4aa05" gracePeriod=30
Oct 30 22:32:08.064687 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.064649    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"]
Oct 30 22:32:08.113852 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.113806    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-b8b85"]
Oct 30 22:32:08.185174 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.185136    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"]
Oct 30 22:32:08.185637 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.185621    5586 cpu_manager.go:414] "containerMap: removing container" podUID="871270f6-45aa-4e80-bcf2-13c77b76baae" containerName="revoke-keys"
Oct 30 22:32:08.185694 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.185638    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="871270f6-45aa-4e80-bcf2-13c77b76baae" containerName="revoke-keys"
Oct 30 22:32:08.185694 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.185662    5586 cpu_manager.go:414] "containerMap: removing container" podUID="51759e2d-609a-4696-b9e2-fe934df66643" containerName="main"
Oct 30 22:32:08.185694 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.185667    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51759e2d-609a-4696-b9e2-fe934df66643" containerName="main"
Oct 30 22:32:08.185792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.185749    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="51759e2d-609a-4696-b9e2-fe934df66643" containerName="main"
Oct 30 22:32:08.185792 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.185762    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="871270f6-45aa-4e80-bcf2-13c77b76baae" containerName="revoke-keys"
Oct 30 22:32:08.190087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.190068    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"
Oct 30 22:32:08.245094 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.245062    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:32:08.245429 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.245311    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:32:08.248087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.248058    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"]
Oct 30 22:32:08.281764 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.281723    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5822a27-d455-4f22-875c-7483c087c8c9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-pxn9p\" (UID: \"a5822a27-d455-4f22-875c-7483c087c8c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"
Oct 30 22:32:08.281940 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.281792    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh9c2\" (UniqueName: \"kubernetes.io/projected/a5822a27-d455-4f22-875c-7483c087c8c9-kube-api-access-vh9c2\") pod \"maas-api-revoke-sub-3d4156f1-pxn9p\" (UID: \"a5822a27-d455-4f22-875c-7483c087c8c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"
Oct 30 22:32:08.281940 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.281885    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5822a27-d455-4f22-875c-7483c087c8c9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-pxn9p\" (UID: \"a5822a27-d455-4f22-875c-7483c087c8c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"
Oct 30 22:32:08.332880 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.332791    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:32:08.332880 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.332848    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:32:08.382500 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.382463    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh9c2\" (UniqueName: \"kubernetes.io/projected/a5822a27-d455-4f22-875c-7483c087c8c9-kube-api-access-vh9c2\") pod \"maas-api-revoke-sub-3d4156f1-pxn9p\" (UID: \"a5822a27-d455-4f22-875c-7483c087c8c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"
Oct 30 22:32:08.382690 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.382512    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5822a27-d455-4f22-875c-7483c087c8c9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-pxn9p\" (UID: \"a5822a27-d455-4f22-875c-7483c087c8c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"
Oct 30 22:32:08.382690 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.382587    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5822a27-d455-4f22-875c-7483c087c8c9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-pxn9p\" (UID: \"a5822a27-d455-4f22-875c-7483c087c8c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"
Oct 30 22:32:08.383195 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.383176    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5822a27-d455-4f22-875c-7483c087c8c9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-pxn9p\" (UID: \"a5822a27-d455-4f22-875c-7483c087c8c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"
Oct 30 22:32:08.385022 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.384989    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5822a27-d455-4f22-875c-7483c087c8c9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-pxn9p\" (UID: \"a5822a27-d455-4f22-875c-7483c087c8c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"
Oct 30 22:32:08.448809 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.448766    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh9c2\" (UniqueName: \"kubernetes.io/projected/a5822a27-d455-4f22-875c-7483c087c8c9-kube-api-access-vh9c2\") pod \"maas-api-revoke-sub-3d4156f1-pxn9p\" (UID: \"a5822a27-d455-4f22-875c-7483c087c8c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"
Oct 30 22:32:08.502724 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.502680    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"
Oct 30 22:32:08.879978 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.879947    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"]
Oct 30 22:32:08.883054 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:32:08.883029    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5822a27_d455_4f22_875c_7483c087c8c9.slice/crio-f65540e4e950e7b6ef0ed96a3254cdcc94349f37522b1bc7a5030893023fc17a WatchSource:0}: Error finding container f65540e4e950e7b6ef0ed96a3254cdcc94349f37522b1bc7a5030893023fc17a: Status 404 returned error can't find the container with id f65540e4e950e7b6ef0ed96a3254cdcc94349f37522b1bc7a5030893023fc17a
Oct 30 22:32:08.926577 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:08.926545    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p" event={"ID":"a5822a27-d455-4f22-875c-7483c087c8c9","Type":"ContainerStarted","Data":"f65540e4e950e7b6ef0ed96a3254cdcc94349f37522b1bc7a5030893023fc17a"}
Oct 30 22:32:09.446907 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:09.446874    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fed66b60-0f21-4433-b567-449b3c05043d" path="/var/lib/kubelet/pods/fed66b60-0f21-4433-b567-449b3c05043d/volumes"
Oct 30 22:32:09.934303 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:09.934266    5586 generic.go:356] "Generic (PLEG): container finished" podID="a5822a27-d455-4f22-875c-7483c087c8c9" containerID="fe9e34d58d06ee4f6adb72772bf71ac04eed236b2be1a21050e19d8f840cad23" exitCode=0
Oct 30 22:32:09.934498 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:09.934346    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p" event={"ID":"a5822a27-d455-4f22-875c-7483c087c8c9","Type":"ContainerDied","Data":"fe9e34d58d06ee4f6adb72772bf71ac04eed236b2be1a21050e19d8f840cad23"}
Oct 30 22:32:11.079377 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:11.079350    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"
Oct 30 22:32:11.211108 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:11.211017    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5822a27-d455-4f22-875c-7483c087c8c9-kube-api-access-vh9c2\" (UniqueName: \"kubernetes.io/projected/a5822a27-d455-4f22-875c-7483c087c8c9-kube-api-access-vh9c2\") pod \"a5822a27-d455-4f22-875c-7483c087c8c9\" (UID: \"a5822a27-d455-4f22-875c-7483c087c8c9\") "
Oct 30 22:32:11.211108 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:11.211101    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a5822a27-d455-4f22-875c-7483c087c8c9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5822a27-d455-4f22-875c-7483c087c8c9-maas-api-xks-ca\") pod \"a5822a27-d455-4f22-875c-7483c087c8c9\" (UID: \"a5822a27-d455-4f22-875c-7483c087c8c9\") "
Oct 30 22:32:11.211408 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:11.211159    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5822a27-d455-4f22-875c-7483c087c8c9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5822a27-d455-4f22-875c-7483c087c8c9-maas-api-openshift-ca\") pod \"a5822a27-d455-4f22-875c-7483c087c8c9\" (UID: \"a5822a27-d455-4f22-875c-7483c087c8c9\") "
Oct 30 22:32:11.211583 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:11.211547    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5822a27-d455-4f22-875c-7483c087c8c9-maas-api-openshift-ca" pod "a5822a27-d455-4f22-875c-7483c087c8c9" (UID: "a5822a27-d455-4f22-875c-7483c087c8c9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:32:11.211685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:11.211666    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a5822a27-d455-4f22-875c-7483c087c8c9-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:11.213566 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:11.213536    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5822a27-d455-4f22-875c-7483c087c8c9-maas-api-xks-ca" pod "a5822a27-d455-4f22-875c-7483c087c8c9" (UID: "a5822a27-d455-4f22-875c-7483c087c8c9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:32:11.213688 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:11.213599    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5822a27-d455-4f22-875c-7483c087c8c9-kube-api-access-vh9c2" pod "a5822a27-d455-4f22-875c-7483c087c8c9" (UID: "a5822a27-d455-4f22-875c-7483c087c8c9"). InnerVolumeSpecName "kube-api-access-vh9c2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:32:11.312594 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:11.312555    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh9c2\" (UniqueName: \"kubernetes.io/projected/a5822a27-d455-4f22-875c-7483c087c8c9-kube-api-access-vh9c2\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:11.312594 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:11.312589    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a5822a27-d455-4f22-875c-7483c087c8c9-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:11.947314 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:11.947275    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p" event={"ID":"a5822a27-d455-4f22-875c-7483c087c8c9","Type":"ContainerDied","Data":"f65540e4e950e7b6ef0ed96a3254cdcc94349f37522b1bc7a5030893023fc17a"}
Oct 30 22:32:11.947314 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:11.947300    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"
Oct 30 22:32:11.947492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:11.947338    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f65540e4e950e7b6ef0ed96a3254cdcc94349f37522b1bc7a5030893023fc17a"
Oct 30 22:32:12.047985 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.047938    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"]
Oct 30 22:32:12.089178 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.089140    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-n6hfm"]
Oct 30 22:32:12.191377 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.191177    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:32:12.324129 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324082    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-data\") pod \"68b70377-92ae-4856-b850-9d8a817f3ccc\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") "
Oct 30 22:32:12.324330 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324150    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68b70377-92ae-4856-b850-9d8a817f3ccc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68b70377-92ae-4856-b850-9d8a817f3ccc-istiod-ca-cert\") pod \"68b70377-92ae-4856-b850-9d8a817f3ccc\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") "
Oct 30 22:32:12.324330 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324191    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68b70377-92ae-4856-b850-9d8a817f3ccc-kube-api-access-mfxlt\" (UniqueName: \"kubernetes.io/projected/68b70377-92ae-4856-b850-9d8a817f3ccc-kube-api-access-mfxlt\") pod \"68b70377-92ae-4856-b850-9d8a817f3ccc\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") "
Oct 30 22:32:12.324330 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324227    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-workload-certs\") pod \"68b70377-92ae-4856-b850-9d8a817f3ccc\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") "
Oct 30 22:32:12.324330 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324299    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-workload-socket\") pod \"68b70377-92ae-4856-b850-9d8a817f3ccc\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") "
Oct 30 22:32:12.324330 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324328    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-credential-socket\") pod \"68b70377-92ae-4856-b850-9d8a817f3ccc\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") "
Oct 30 22:32:12.324652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324361    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-token\" (UniqueName: \"kubernetes.io/projected/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-token\") pod \"68b70377-92ae-4856-b850-9d8a817f3ccc\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") "
Oct 30 22:32:12.324652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324410    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-envoy\") pod \"68b70377-92ae-4856-b850-9d8a817f3ccc\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") "
Oct 30 22:32:12.324652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324483    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-podinfo\") pod \"68b70377-92ae-4856-b850-9d8a817f3ccc\" (UID: \"68b70377-92ae-4856-b850-9d8a817f3ccc\") "
Oct 30 22:32:12.324652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324469    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-data" pod "68b70377-92ae-4856-b850-9d8a817f3ccc" (UID: "68b70377-92ae-4856-b850-9d8a817f3ccc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:32:12.324652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324580    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-workload-certs" pod "68b70377-92ae-4856-b850-9d8a817f3ccc" (UID: "68b70377-92ae-4856-b850-9d8a817f3ccc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:32:12.324652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324625    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-workload-socket" pod "68b70377-92ae-4856-b850-9d8a817f3ccc" (UID: "68b70377-92ae-4856-b850-9d8a817f3ccc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:32:12.324652 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324637    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68b70377-92ae-4856-b850-9d8a817f3ccc-istiod-ca-cert" pod "68b70377-92ae-4856-b850-9d8a817f3ccc" (UID: "68b70377-92ae-4856-b850-9d8a817f3ccc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:32:12.325027 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324820    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-credential-socket" pod "68b70377-92ae-4856-b850-9d8a817f3ccc" (UID: "68b70377-92ae-4856-b850-9d8a817f3ccc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:32:12.325027 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324975    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68b70377-92ae-4856-b850-9d8a817f3ccc-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:12.325027 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.324994    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:12.325027 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.325005    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:12.325027 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.325014    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:12.325027 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.325023    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:12.326831 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.326802    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-token" pod "68b70377-92ae-4856-b850-9d8a817f3ccc" (UID: "68b70377-92ae-4856-b850-9d8a817f3ccc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:32:12.326831 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.326813    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-podinfo" pod "68b70377-92ae-4856-b850-9d8a817f3ccc" (UID: "68b70377-92ae-4856-b850-9d8a817f3ccc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:32:12.327046 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.326855    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68b70377-92ae-4856-b850-9d8a817f3ccc-kube-api-access-mfxlt" pod "68b70377-92ae-4856-b850-9d8a817f3ccc" (UID: "68b70377-92ae-4856-b850-9d8a817f3ccc"). InnerVolumeSpecName "kube-api-access-mfxlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:32:12.327120 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.327075    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-envoy" pod "68b70377-92ae-4856-b850-9d8a817f3ccc" (UID: "68b70377-92ae-4856-b850-9d8a817f3ccc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:32:12.426174 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.426129    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfxlt\" (UniqueName: \"kubernetes.io/projected/68b70377-92ae-4856-b850-9d8a817f3ccc-kube-api-access-mfxlt\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:12.426174 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.426168    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:12.426174 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.426179    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:12.426466 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.426188    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68b70377-92ae-4856-b850-9d8a817f3ccc-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:32:12.956759 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.956724    5586 generic.go:356] "Generic (PLEG): container finished" podID="68b70377-92ae-4856-b850-9d8a817f3ccc" containerID="d1909c8d41b6f1469e709f52b0cb8456bc7efdbdaa500e7079b416615dc4aa05" exitCode=0
Oct 30 22:32:12.956984 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.956786    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"
Oct 30 22:32:12.956984 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.956805    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f" event={"ID":"68b70377-92ae-4856-b850-9d8a817f3ccc","Type":"ContainerDied","Data":"d1909c8d41b6f1469e709f52b0cb8456bc7efdbdaa500e7079b416615dc4aa05"}
Oct 30 22:32:12.956984 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.956887    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f" event={"ID":"68b70377-92ae-4856-b850-9d8a817f3ccc","Type":"ContainerDied","Data":"dfff112a4c29dc8da4d2287a5f46002e4519bff7b9c0cbcd06d23946b8d2fbf6"}
Oct 30 22:32:12.956984 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.956910    5586 scope.go:122] "RemoveContainer" containerID="d1909c8d41b6f1469e709f52b0cb8456bc7efdbdaa500e7079b416615dc4aa05"
Oct 30 22:32:12.968350 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.968329    5586 scope.go:122] "RemoveContainer" containerID="1a5668084365f9ab859a48824ca33ee495663624572f72d3f2cdcff54ab6d8fa"
Oct 30 22:32:12.977847 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.977825    5586 scope.go:122] "RemoveContainer" containerID="d1909c8d41b6f1469e709f52b0cb8456bc7efdbdaa500e7079b416615dc4aa05"
Oct 30 22:32:12.978174 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:32:12.978155    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1909c8d41b6f1469e709f52b0cb8456bc7efdbdaa500e7079b416615dc4aa05\": container with ID starting with d1909c8d41b6f1469e709f52b0cb8456bc7efdbdaa500e7079b416615dc4aa05 not found: ID does not exist" containerID="d1909c8d41b6f1469e709f52b0cb8456bc7efdbdaa500e7079b416615dc4aa05"
Oct 30 22:32:12.978219 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.978184    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1909c8d41b6f1469e709f52b0cb8456bc7efdbdaa500e7079b416615dc4aa05"} err="failed to get container status \"d1909c8d41b6f1469e709f52b0cb8456bc7efdbdaa500e7079b416615dc4aa05\": rpc error: code = NotFound desc = could not find container \"d1909c8d41b6f1469e709f52b0cb8456bc7efdbdaa500e7079b416615dc4aa05\": container with ID starting with d1909c8d41b6f1469e709f52b0cb8456bc7efdbdaa500e7079b416615dc4aa05 not found: ID does not exist"
Oct 30 22:32:12.978219 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.978206    5586 scope.go:122] "RemoveContainer" containerID="1a5668084365f9ab859a48824ca33ee495663624572f72d3f2cdcff54ab6d8fa"
Oct 30 22:32:12.978506 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:32:12.978488    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a5668084365f9ab859a48824ca33ee495663624572f72d3f2cdcff54ab6d8fa\": container with ID starting with 1a5668084365f9ab859a48824ca33ee495663624572f72d3f2cdcff54ab6d8fa not found: ID does not exist" containerID="1a5668084365f9ab859a48824ca33ee495663624572f72d3f2cdcff54ab6d8fa"
Oct 30 22:32:12.978544 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:12.978513    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a5668084365f9ab859a48824ca33ee495663624572f72d3f2cdcff54ab6d8fa"} err="failed to get container status \"1a5668084365f9ab859a48824ca33ee495663624572f72d3f2cdcff54ab6d8fa\": rpc error: code = NotFound desc = could not find container \"1a5668084365f9ab859a48824ca33ee495663624572f72d3f2cdcff54ab6d8fa\": container with ID starting with 1a5668084365f9ab859a48824ca33ee495663624572f72d3f2cdcff54ab6d8fa not found: ID does not exist"
Oct 30 22:32:13.170836 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:13.170798    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"]
Oct 30 22:32:13.220589 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:13.220504    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-5b7354-openshift-default-dc46bff89-f7w9f"]
Oct 30 22:32:13.447487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:13.447439    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68613646-0636-483c-90cb-e73f7a6bfc8d" path="/var/lib/kubelet/pods/68613646-0636-483c-90cb-e73f7a6bfc8d/volumes"
Oct 30 22:32:13.447995 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:13.447973    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68b70377-92ae-4856-b850-9d8a817f3ccc" path="/var/lib/kubelet/pods/68b70377-92ae-4856-b850-9d8a817f3ccc/volumes"
Oct 30 22:32:18.360244 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:32:18.360187    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:32:19.526970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:19.526927    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:21.042250 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:21.042193    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:21.492892 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:21.492860    5586 scope.go:122] "RemoveContainer" containerID="2cb77808e69752a486fbb8e77cf373e297f199f4c32954f96469710dc31ded57"
Oct 30 22:32:26.883540 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:26.883500    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:35.698111 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:35.698070    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:43.841022 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:43.840975    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:48.360635 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:32:48.360594    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:32:52.047797 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:52.047761    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"]
Oct 30 22:32:52.096744 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:52.096704    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-fczj8"]
Oct 30 22:32:53.446588 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:32:53.446553    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="583359e2-b5c3-48a7-b673-e8eda96b493b" path="/var/lib/kubelet/pods/583359e2-b5c3-48a7-b673-e8eda96b493b/volumes"
Oct 30 22:33:18.360956 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:33:18.360914    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:33:21.530739 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:21.530695    5586 scope.go:122] "RemoveContainer" containerID="e55b776ae42576ed29581a8b863935c129d4d8f3869d51c99408231c575c598e"
Oct 30 22:33:21.540110 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:21.540087    5586 scope.go:122] "RemoveContainer" containerID="ca4a9d7fd7698b8b6d00b524b383196af1d822066c1810e3ae7d266b51bbf436"
Oct 30 22:33:36.048673 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:36.048627    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:40.789889 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.789849    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl"]
Oct 30 22:33:40.790383 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.790320    5586 cpu_manager.go:414] "containerMap: removing container" podUID="a5822a27-d455-4f22-875c-7483c087c8c9" containerName="revoke-keys"
Oct 30 22:33:40.790383 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.790333    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5822a27-d455-4f22-875c-7483c087c8c9" containerName="revoke-keys"
Oct 30 22:33:40.790383 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.790353    5586 cpu_manager.go:414] "containerMap: removing container" podUID="68b70377-92ae-4856-b850-9d8a817f3ccc" containerName="istio-proxy"
Oct 30 22:33:40.790383 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.790358    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68b70377-92ae-4856-b850-9d8a817f3ccc" containerName="istio-proxy"
Oct 30 22:33:40.790383 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.790369    5586 cpu_manager.go:414] "containerMap: removing container" podUID="68b70377-92ae-4856-b850-9d8a817f3ccc" containerName="istio-proxy"
Oct 30 22:33:40.790383 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.790375    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68b70377-92ae-4856-b850-9d8a817f3ccc" containerName="istio-proxy"
Oct 30 22:33:40.790594 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.790460    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5822a27-d455-4f22-875c-7483c087c8c9" containerName="revoke-keys"
Oct 30 22:33:40.790594 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.790471    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="68b70377-92ae-4856-b850-9d8a817f3ccc" containerName="istio-proxy"
Oct 30 22:33:40.790594 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.790478    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="68b70377-92ae-4856-b850-9d8a817f3ccc" containerName="istio-proxy"
Oct 30 22:33:40.795307 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.795287    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl"
Oct 30 22:33:40.845959 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.845919    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl"]
Oct 30 22:33:40.892513 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.892475    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lsrdr\""
Oct 30 22:33:40.899913 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.899884    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1288e96c-85cb-4e3a-a566-ea9253b8307a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-gz4nl\" (UID: \"1288e96c-85cb-4e3a-a566-ea9253b8307a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl"
Oct 30 22:33:40.900050 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:40.899955    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqhkv\" (UniqueName: \"kubernetes.io/projected/1288e96c-85cb-4e3a-a566-ea9253b8307a-kube-api-access-jqhkv\") pod \"kuadrant-operator-controller-manager-678d87766f-gz4nl\" (UID: \"1288e96c-85cb-4e3a-a566-ea9253b8307a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl"
Oct 30 22:33:41.000836 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:41.000792    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1288e96c-85cb-4e3a-a566-ea9253b8307a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-gz4nl\" (UID: \"1288e96c-85cb-4e3a-a566-ea9253b8307a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl"
Oct 30 22:33:41.001035 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:41.000870    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqhkv\" (UniqueName: \"kubernetes.io/projected/1288e96c-85cb-4e3a-a566-ea9253b8307a-kube-api-access-jqhkv\") pod \"kuadrant-operator-controller-manager-678d87766f-gz4nl\" (UID: \"1288e96c-85cb-4e3a-a566-ea9253b8307a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl"
Oct 30 22:33:41.001209 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:41.001189    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1288e96c-85cb-4e3a-a566-ea9253b8307a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-gz4nl\" (UID: \"1288e96c-85cb-4e3a-a566-ea9253b8307a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl"
Oct 30 22:33:41.057634 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:41.057560    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqhkv\" (UniqueName: \"kubernetes.io/projected/1288e96c-85cb-4e3a-a566-ea9253b8307a-kube-api-access-jqhkv\") pod \"kuadrant-operator-controller-manager-678d87766f-gz4nl\" (UID: \"1288e96c-85cb-4e3a-a566-ea9253b8307a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl"
Oct 30 22:33:41.108769 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:41.108730    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl"
Oct 30 22:33:41.589448 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:41.589420    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl"]
Oct 30 22:33:41.592026 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:33:41.591993    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1288e96c_85cb_4e3a_a566_ea9253b8307a.slice/crio-6eb5a081d2cf04c35c0660b4aa4bbdb5c0a6c31250a5195feab6631a33f775df WatchSource:0}: Error finding container 6eb5a081d2cf04c35c0660b4aa4bbdb5c0a6c31250a5195feab6631a33f775df: Status 404 returned error can't find the container with id 6eb5a081d2cf04c35c0660b4aa4bbdb5c0a6c31250a5195feab6631a33f775df
Oct 30 22:33:42.475973 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:42.475933    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl" event={"ID":"1288e96c-85cb-4e3a-a566-ea9253b8307a","Type":"ContainerStarted","Data":"7877100aa95fd0feaaf160c7044452d31e1c95e1445290e696178ed64ad2b675"}
Oct 30 22:33:42.476424 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:42.476105    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl"
Oct 30 22:33:42.476424 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:42.476128    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl" event={"ID":"1288e96c-85cb-4e3a-a566-ea9253b8307a","Type":"ContainerStarted","Data":"6eb5a081d2cf04c35c0660b4aa4bbdb5c0a6c31250a5195feab6631a33f775df"}
Oct 30 22:33:42.696763 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:42.696696    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl" podStartSLOduration=2.696673896 podStartE2EDuration="2.696673896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:33:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:33:42.644740863 +0000 UTC m=+3565.316644803" watchObservedRunningTime="2026-10-30 22:33:42.696673896 +0000 UTC m=+3565.368577835"
Oct 30 22:33:48.360880 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:33:48.360838    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:33:53.486016 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:33:53.485984    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-gz4nl"
Oct 30 22:34:12.451835 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.451787    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"]
Oct 30 22:34:12.455708 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.455688    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"
Oct 30 22:34:12.513548 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.513509    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"]
Oct 30 22:34:12.558890 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.558857    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:34:12.558890 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.558860    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:34:12.558890 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.558887    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:34:12.559151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.558910    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:34:12.566416 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.566394    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9494a779-aacb-4dd1-9e78-0061a552799d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-wklxv\" (UID: \"9494a779-aacb-4dd1-9e78-0061a552799d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"
Oct 30 22:34:12.566470 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.566448    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9494a779-aacb-4dd1-9e78-0061a552799d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-wklxv\" (UID: \"9494a779-aacb-4dd1-9e78-0061a552799d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"
Oct 30 22:34:12.566522 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.566507    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zttpc\" (UniqueName: \"kubernetes.io/projected/9494a779-aacb-4dd1-9e78-0061a552799d-kube-api-access-zttpc\") pod \"maas-api-revoke-sub-d576e68b-wklxv\" (UID: \"9494a779-aacb-4dd1-9e78-0061a552799d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"
Oct 30 22:34:12.667343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.667312    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9494a779-aacb-4dd1-9e78-0061a552799d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-wklxv\" (UID: \"9494a779-aacb-4dd1-9e78-0061a552799d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"
Oct 30 22:34:12.667549 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.667371    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9494a779-aacb-4dd1-9e78-0061a552799d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-wklxv\" (UID: \"9494a779-aacb-4dd1-9e78-0061a552799d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"
Oct 30 22:34:12.667549 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.667494    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zttpc\" (UniqueName: \"kubernetes.io/projected/9494a779-aacb-4dd1-9e78-0061a552799d-kube-api-access-zttpc\") pod \"maas-api-revoke-sub-d576e68b-wklxv\" (UID: \"9494a779-aacb-4dd1-9e78-0061a552799d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"
Oct 30 22:34:12.667988 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.667968    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9494a779-aacb-4dd1-9e78-0061a552799d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-wklxv\" (UID: \"9494a779-aacb-4dd1-9e78-0061a552799d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"
Oct 30 22:34:12.669941 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.669918    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9494a779-aacb-4dd1-9e78-0061a552799d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-wklxv\" (UID: \"9494a779-aacb-4dd1-9e78-0061a552799d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"
Oct 30 22:34:12.725091 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.725016    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zttpc\" (UniqueName: \"kubernetes.io/projected/9494a779-aacb-4dd1-9e78-0061a552799d-kube-api-access-zttpc\") pod \"maas-api-revoke-sub-d576e68b-wklxv\" (UID: \"9494a779-aacb-4dd1-9e78-0061a552799d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"
Oct 30 22:34:12.768771 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:12.768729    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"
Oct 30 22:34:13.146512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:13.146484    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"]
Oct 30 22:34:13.148565 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:34:13.148529    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9494a779_aacb_4dd1_9e78_0061a552799d.slice/crio-575b9733a749d90c9bd83916a7a5aa32cda98671d15003cd0da6b7f91720b0f6 WatchSource:0}: Error finding container 575b9733a749d90c9bd83916a7a5aa32cda98671d15003cd0da6b7f91720b0f6: Status 404 returned error can't find the container with id 575b9733a749d90c9bd83916a7a5aa32cda98671d15003cd0da6b7f91720b0f6
Oct 30 22:34:13.679718 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:13.679676    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv" event={"ID":"9494a779-aacb-4dd1-9e78-0061a552799d","Type":"ContainerStarted","Data":"c2fc9c8868b926ea6cf099877aed3901af9631aa46298cc6ec0e619aafb80115"}
Oct 30 22:34:13.680213 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:13.679782    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv" event={"ID":"9494a779-aacb-4dd1-9e78-0061a552799d","Type":"ContainerStarted","Data":"575b9733a749d90c9bd83916a7a5aa32cda98671d15003cd0da6b7f91720b0f6"}
Oct 30 22:34:13.796694 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:13.796637    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv" podStartSLOduration=1.796622532 podStartE2EDuration="1.796622532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:34:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:34:13.794670014 +0000 UTC m=+3596.466573946" watchObservedRunningTime="2026-10-30 22:34:13.796622532 +0000 UTC m=+3596.468526530"
Oct 30 22:34:14.687554 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:14.687517    5586 generic.go:356] "Generic (PLEG): container finished" podID="9494a779-aacb-4dd1-9e78-0061a552799d" containerID="c2fc9c8868b926ea6cf099877aed3901af9631aa46298cc6ec0e619aafb80115" exitCode=0
Oct 30 22:34:14.688027 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:14.687600    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv" event={"ID":"9494a779-aacb-4dd1-9e78-0061a552799d","Type":"ContainerDied","Data":"c2fc9c8868b926ea6cf099877aed3901af9631aa46298cc6ec0e619aafb80115"}
Oct 30 22:34:15.833405 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:15.833375    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"
Oct 30 22:34:15.999260 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:15.999166    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9494a779-aacb-4dd1-9e78-0061a552799d-kube-api-access-zttpc\" (UniqueName: \"kubernetes.io/projected/9494a779-aacb-4dd1-9e78-0061a552799d-kube-api-access-zttpc\") pod \"9494a779-aacb-4dd1-9e78-0061a552799d\" (UID: \"9494a779-aacb-4dd1-9e78-0061a552799d\") "
Oct 30 22:34:15.999437 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:15.999335    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9494a779-aacb-4dd1-9e78-0061a552799d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9494a779-aacb-4dd1-9e78-0061a552799d-maas-api-openshift-ca\") pod \"9494a779-aacb-4dd1-9e78-0061a552799d\" (UID: \"9494a779-aacb-4dd1-9e78-0061a552799d\") "
Oct 30 22:34:15.999490 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:15.999453    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9494a779-aacb-4dd1-9e78-0061a552799d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9494a779-aacb-4dd1-9e78-0061a552799d-maas-api-xks-ca\") pod \"9494a779-aacb-4dd1-9e78-0061a552799d\" (UID: \"9494a779-aacb-4dd1-9e78-0061a552799d\") "
Oct 30 22:34:15.999693 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:15.999666    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9494a779-aacb-4dd1-9e78-0061a552799d-maas-api-openshift-ca" pod "9494a779-aacb-4dd1-9e78-0061a552799d" (UID: "9494a779-aacb-4dd1-9e78-0061a552799d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:34:15.999796 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:15.999779    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9494a779-aacb-4dd1-9e78-0061a552799d-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:16.001413 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:16.001388    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9494a779-aacb-4dd1-9e78-0061a552799d-maas-api-xks-ca" pod "9494a779-aacb-4dd1-9e78-0061a552799d" (UID: "9494a779-aacb-4dd1-9e78-0061a552799d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:34:16.001513 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:16.001458    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9494a779-aacb-4dd1-9e78-0061a552799d-kube-api-access-zttpc" pod "9494a779-aacb-4dd1-9e78-0061a552799d" (UID: "9494a779-aacb-4dd1-9e78-0061a552799d"). InnerVolumeSpecName "kube-api-access-zttpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:16.100520 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:16.100478    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9494a779-aacb-4dd1-9e78-0061a552799d-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:16.100520 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:16.100513    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zttpc\" (UniqueName: \"kubernetes.io/projected/9494a779-aacb-4dd1-9e78-0061a552799d-kube-api-access-zttpc\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:16.700869 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:16.700835    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"
Oct 30 22:34:16.700869 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:16.700862    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv" event={"ID":"9494a779-aacb-4dd1-9e78-0061a552799d","Type":"ContainerDied","Data":"575b9733a749d90c9bd83916a7a5aa32cda98671d15003cd0da6b7f91720b0f6"}
Oct 30 22:34:16.701145 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:16.700908    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="575b9733a749d90c9bd83916a7a5aa32cda98671d15003cd0da6b7f91720b0f6"
Oct 30 22:34:18.050538 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:18.050497    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"]
Oct 30 22:34:18.099882 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:18.099846    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-5nl8t"]
Oct 30 22:34:18.170102 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:18.170074    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:34:18.174817 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:18.174789    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:34:18.181312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:18.181286    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:34:18.186009 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:18.185988    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:34:18.361045 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:34:18.360962    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:34:19.446315 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:19.446282    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a5d19a2-e6aa-48ec-9f41-0a0670268e0b" path="/var/lib/kubelet/pods/7a5d19a2-e6aa-48ec-9f41-0a0670268e0b/volumes"
Oct 30 22:34:21.568287 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:21.568247    5586 scope.go:122] "RemoveContainer" containerID="8786359d7ca0576be09649f41fff05cef6c2c27ed06cb2d4cf4b50333ddbb2e7"
Oct 30 22:34:21.577519 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:21.577497    5586 scope.go:122] "RemoveContainer" containerID="cb1f7c3bd53e67225f37e007b52a34d065ea13bc9f222de482b95089838c2d63"
Oct 30 22:34:29.049916 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:29.049834    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"]
Oct 30 22:34:29.054300 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:29.054274    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-d5lc7"]
Oct 30 22:34:29.447310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:29.447274    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6" path="/var/lib/kubelet/pods/a03eeb81-ee85-422b-bc49-a8cc1c9aa8a6/volumes"
Oct 30 22:34:36.343294 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:36.343222    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:41.527366 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.527334    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"]
Oct 30 22:34:41.527788 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.527767    5586 cpu_manager.go:414] "containerMap: removing container" podUID="9494a779-aacb-4dd1-9e78-0061a552799d" containerName="revoke-keys"
Oct 30 22:34:41.527788 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.527782    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9494a779-aacb-4dd1-9e78-0061a552799d" containerName="revoke-keys"
Oct 30 22:34:41.527877 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.527847    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="9494a779-aacb-4dd1-9e78-0061a552799d" containerName="revoke-keys"
Oct 30 22:34:41.531381 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.531361    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"
Oct 30 22:34:41.585841 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.585792    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:34:41.589396 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.589369    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"]
Oct 30 22:34:41.635030 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.635001    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:34:41.635221 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.635062    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:34:41.635221 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.635072    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:34:41.712206 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.712162    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd4cr\" (UniqueName: \"kubernetes.io/projected/ede83587-9e6f-486a-ab06-d7d87f985788-kube-api-access-cd4cr\") pod \"maas-api-revoke-sub-d2bc93bc-d7689\" (UID: \"ede83587-9e6f-486a-ab06-d7d87f985788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"
Oct 30 22:34:41.712206 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.712212    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ede83587-9e6f-486a-ab06-d7d87f985788-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-d7689\" (UID: \"ede83587-9e6f-486a-ab06-d7d87f985788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"
Oct 30 22:34:41.712455 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.712313    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ede83587-9e6f-486a-ab06-d7d87f985788-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-d7689\" (UID: \"ede83587-9e6f-486a-ab06-d7d87f985788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"
Oct 30 22:34:41.813351 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.813258    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ede83587-9e6f-486a-ab06-d7d87f985788-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-d7689\" (UID: \"ede83587-9e6f-486a-ab06-d7d87f985788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"
Oct 30 22:34:41.813516 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.813386    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd4cr\" (UniqueName: \"kubernetes.io/projected/ede83587-9e6f-486a-ab06-d7d87f985788-kube-api-access-cd4cr\") pod \"maas-api-revoke-sub-d2bc93bc-d7689\" (UID: \"ede83587-9e6f-486a-ab06-d7d87f985788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"
Oct 30 22:34:41.813516 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.813425    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ede83587-9e6f-486a-ab06-d7d87f985788-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-d7689\" (UID: \"ede83587-9e6f-486a-ab06-d7d87f985788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"
Oct 30 22:34:41.813918 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.813898    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ede83587-9e6f-486a-ab06-d7d87f985788-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-d7689\" (UID: \"ede83587-9e6f-486a-ab06-d7d87f985788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"
Oct 30 22:34:41.815815 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.815795    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ede83587-9e6f-486a-ab06-d7d87f985788-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-d7689\" (UID: \"ede83587-9e6f-486a-ab06-d7d87f985788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"
Oct 30 22:34:41.871354 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:41.871327    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd4cr\" (UniqueName: \"kubernetes.io/projected/ede83587-9e6f-486a-ab06-d7d87f985788-kube-api-access-cd4cr\") pod \"maas-api-revoke-sub-d2bc93bc-d7689\" (UID: \"ede83587-9e6f-486a-ab06-d7d87f985788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"
Oct 30 22:34:42.144734 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:42.144640    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"
Oct 30 22:34:42.521875 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:42.521850    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"]
Oct 30 22:34:42.524063 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:34:42.524023    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podede83587_9e6f_486a_ab06_d7d87f985788.slice/crio-2dcdc7ecde27a670b723f714bec12c8929983fd1495aceddc42471e572bbc6d5 WatchSource:0}: Error finding container 2dcdc7ecde27a670b723f714bec12c8929983fd1495aceddc42471e572bbc6d5: Status 404 returned error can't find the container with id 2dcdc7ecde27a670b723f714bec12c8929983fd1495aceddc42471e572bbc6d5
Oct 30 22:34:42.889574 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:42.889543    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689" event={"ID":"ede83587-9e6f-486a-ab06-d7d87f985788","Type":"ContainerStarted","Data":"946a1cd2eb61e8e3ed03526d4f0c12c33b5a7bdda07a97038e83f422e48631eb"}
Oct 30 22:34:42.890032 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:42.889592    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689" event={"ID":"ede83587-9e6f-486a-ab06-d7d87f985788","Type":"ContainerStarted","Data":"2dcdc7ecde27a670b723f714bec12c8929983fd1495aceddc42471e572bbc6d5"}
Oct 30 22:34:43.006607 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:43.006545    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689" podStartSLOduration=2.006529271 podStartE2EDuration="2.006529271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:34:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:34:43.005551813 +0000 UTC m=+3625.677455748" watchObservedRunningTime="2026-10-30 22:34:43.006529271 +0000 UTC m=+3625.678433205"
Oct 30 22:34:43.896542 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:43.896503    5586 generic.go:356] "Generic (PLEG): container finished" podID="ede83587-9e6f-486a-ab06-d7d87f985788" containerID="946a1cd2eb61e8e3ed03526d4f0c12c33b5a7bdda07a97038e83f422e48631eb" exitCode=0
Oct 30 22:34:43.897022 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:43.896588    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689" event={"ID":"ede83587-9e6f-486a-ab06-d7d87f985788","Type":"ContainerDied","Data":"946a1cd2eb61e8e3ed03526d4f0c12c33b5a7bdda07a97038e83f422e48631eb"}
Oct 30 22:34:45.043801 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:45.043770    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"
Oct 30 22:34:45.147627 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:45.147594    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ede83587-9e6f-486a-ab06-d7d87f985788-kube-api-access-cd4cr\" (UniqueName: \"kubernetes.io/projected/ede83587-9e6f-486a-ab06-d7d87f985788-kube-api-access-cd4cr\") pod \"ede83587-9e6f-486a-ab06-d7d87f985788\" (UID: \"ede83587-9e6f-486a-ab06-d7d87f985788\") "
Oct 30 22:34:45.147824 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:45.147636    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ede83587-9e6f-486a-ab06-d7d87f985788-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ede83587-9e6f-486a-ab06-d7d87f985788-maas-api-xks-ca\") pod \"ede83587-9e6f-486a-ab06-d7d87f985788\" (UID: \"ede83587-9e6f-486a-ab06-d7d87f985788\") "
Oct 30 22:34:45.147824 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:45.147685    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ede83587-9e6f-486a-ab06-d7d87f985788-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ede83587-9e6f-486a-ab06-d7d87f985788-maas-api-openshift-ca\") pod \"ede83587-9e6f-486a-ab06-d7d87f985788\" (UID: \"ede83587-9e6f-486a-ab06-d7d87f985788\") "
Oct 30 22:34:45.148113 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:45.148086    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ede83587-9e6f-486a-ab06-d7d87f985788-maas-api-openshift-ca" pod "ede83587-9e6f-486a-ab06-d7d87f985788" (UID: "ede83587-9e6f-486a-ab06-d7d87f985788"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:34:45.149731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:45.149708    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ede83587-9e6f-486a-ab06-d7d87f985788-maas-api-xks-ca" pod "ede83587-9e6f-486a-ab06-d7d87f985788" (UID: "ede83587-9e6f-486a-ab06-d7d87f985788"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:34:45.149864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:45.149841    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ede83587-9e6f-486a-ab06-d7d87f985788-kube-api-access-cd4cr" pod "ede83587-9e6f-486a-ab06-d7d87f985788" (UID: "ede83587-9e6f-486a-ab06-d7d87f985788"). InnerVolumeSpecName "kube-api-access-cd4cr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:45.248787 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:45.248749    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ede83587-9e6f-486a-ab06-d7d87f985788-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:45.248787 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:45.248780    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cd4cr\" (UniqueName: \"kubernetes.io/projected/ede83587-9e6f-486a-ab06-d7d87f985788-kube-api-access-cd4cr\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:45.248787 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:45.248791    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ede83587-9e6f-486a-ab06-d7d87f985788-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:45.909985 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:45.909944    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689" event={"ID":"ede83587-9e6f-486a-ab06-d7d87f985788","Type":"ContainerDied","Data":"2dcdc7ecde27a670b723f714bec12c8929983fd1495aceddc42471e572bbc6d5"}
Oct 30 22:34:45.910171 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:45.910015    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2dcdc7ecde27a670b723f714bec12c8929983fd1495aceddc42471e572bbc6d5"
Oct 30 22:34:45.910171 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:45.909956    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"
Oct 30 22:34:48.361689 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:34:48.361657    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:34:54.142501 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:54.142455    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:54.831966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:54.831929    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:55.065220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:55.065172    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:55.414564 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:55.414525    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:55.570469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:34:55.570423    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:08.958804 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:08.958764    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:09.594634 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:09.594592    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:12.167644 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:12.167606    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:12.347672 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:12.347635    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:15.250152 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:15.250091    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:15.496425 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:15.496382    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:18.362015 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:35:18.361978    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:35:20.194829 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:20.194787    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:21.605174 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:21.605134    5586 scope.go:122] "RemoveContainer" containerID="748934ee75c2486beb75cf0de5bc6f16c97b601d0bf745c35c4f6d96ed10553b"
Oct 30 22:35:21.615193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:21.615171    5586 scope.go:122] "RemoveContainer" containerID="b27e8d9111f97c801e41a801e247680d33b6cd33d4c060fff08b5bb267706a29"
Oct 30 22:35:25.040726 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:25.040691    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:30.020696 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:30.020665    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:33.773525 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:33.773490    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:34.447750 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:34.447717    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:34.694515 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:34.694461    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:35.053328 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:35.053285    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:36.712941 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:36.712898    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:38.917151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:38.917099    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:39.906268 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:39.906206    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:44.794117 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:44.794070    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:48.363606 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:35:48.363565    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:35:49.584143 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:49.584103    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:54.486848 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:54.486810    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:59.269614 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:35:59.269527    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:04.191139 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:04.191097    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:08.982432 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:08.982366    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:13.811197 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:13.811155    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:18.361527 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:36:18.361487    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:36:18.595736 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:18.595693    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:21.646509 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:21.646475    5586 scope.go:122] "RemoveContainer" containerID="e78763107f8bffaa314cf2a2390a8320d07f366ca000f5c0db19b122e99a13b0"
Oct 30 22:36:21.655660 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:21.655635    5586 scope.go:122] "RemoveContainer" containerID="4fcca0165c5893a257ca27a1eb5f516a8e220be5a611b5edd6018ad0545bd781"
Oct 30 22:36:22.887635 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:22.887601    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"]
Oct 30 22:36:22.888073 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:22.888058    5586 cpu_manager.go:414] "containerMap: removing container" podUID="ede83587-9e6f-486a-ab06-d7d87f985788" containerName="revoke-keys"
Oct 30 22:36:22.888119 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:22.888074    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ede83587-9e6f-486a-ab06-d7d87f985788" containerName="revoke-keys"
Oct 30 22:36:22.888197 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:22.888187    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="ede83587-9e6f-486a-ab06-d7d87f985788" containerName="revoke-keys"
Oct 30 22:36:22.891438 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:22.891422    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"
Oct 30 22:36:22.929092 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:22.929050    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e1e6bd7b-j57zm\" (UID: \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"
Oct 30 22:36:22.929332 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:22.929188    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e1e6bd7b-j57zm\" (UID: \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"
Oct 30 22:36:22.929332 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:22.929244    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9m2m\" (UniqueName: \"kubernetes.io/projected/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-kube-api-access-p9m2m\") pod \"maas-api-revoke-sub-e1e6bd7b-j57zm\" (UID: \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"
Oct 30 22:36:22.951492 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:22.951453    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"]
Oct 30 22:36:22.996039 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:22.995996    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:36:22.996039 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:22.996008    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:36:22.996293 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:22.996046    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:36:22.996293 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:22.996143    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:36:23.030312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:23.030273    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e1e6bd7b-j57zm\" (UID: \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"
Oct 30 22:36:23.030312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:23.030319    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9m2m\" (UniqueName: \"kubernetes.io/projected/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-kube-api-access-p9m2m\") pod \"maas-api-revoke-sub-e1e6bd7b-j57zm\" (UID: \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"
Oct 30 22:36:23.030606 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:23.030382    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e1e6bd7b-j57zm\" (UID: \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"
Oct 30 22:36:23.030964 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:23.030939    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e1e6bd7b-j57zm\" (UID: \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"
Oct 30 22:36:23.032700 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:23.032682    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e1e6bd7b-j57zm\" (UID: \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"
Oct 30 22:36:23.088311 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:23.088268    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9m2m\" (UniqueName: \"kubernetes.io/projected/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-kube-api-access-p9m2m\") pod \"maas-api-revoke-sub-e1e6bd7b-j57zm\" (UID: \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"
Oct 30 22:36:23.215658 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:23.215616    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"
Oct 30 22:36:23.402906 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:23.402866    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:23.589734 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:23.589702    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"]
Oct 30 22:36:23.592299 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:36:23.592264    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80f6fd3f_54ba_482f_af68_3a10abc3ab5c.slice/crio-b4bb9fc5e20a7114ee6eeba9d94f911bed794a6a73001bb5c90cb9b3bb5b2fe2 WatchSource:0}: Error finding container b4bb9fc5e20a7114ee6eeba9d94f911bed794a6a73001bb5c90cb9b3bb5b2fe2: Status 404 returned error can't find the container with id b4bb9fc5e20a7114ee6eeba9d94f911bed794a6a73001bb5c90cb9b3bb5b2fe2
Oct 30 22:36:24.503811 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:24.503783    5586 generic.go:356] "Generic (PLEG): container finished" podID="80f6fd3f-54ba-482f-af68-3a10abc3ab5c" containerID="4a1473931c3270bd1a168e19f895c9d0740eee70d750f17dfc4817cd451ac9f1" exitCode=0
Oct 30 22:36:24.504153 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:24.503867    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm" event={"ID":"80f6fd3f-54ba-482f-af68-3a10abc3ab5c","Type":"ContainerDied","Data":"4a1473931c3270bd1a168e19f895c9d0740eee70d750f17dfc4817cd451ac9f1"}
Oct 30 22:36:24.504153 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:24.503922    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm" event={"ID":"80f6fd3f-54ba-482f-af68-3a10abc3ab5c","Type":"ContainerStarted","Data":"b4bb9fc5e20a7114ee6eeba9d94f911bed794a6a73001bb5c90cb9b3bb5b2fe2"}
Oct 30 22:36:25.652551 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:25.652525    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"
Oct 30 22:36:25.758745 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:25.758696    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-maas-api-xks-ca\") pod \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\" (UID: \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\") "
Oct 30 22:36:25.758970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:25.758755    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-maas-api-openshift-ca\") pod \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\" (UID: \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\") "
Oct 30 22:36:25.758970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:25.758777    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-kube-api-access-p9m2m\" (UniqueName: \"kubernetes.io/projected/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-kube-api-access-p9m2m\") pod \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\" (UID: \"80f6fd3f-54ba-482f-af68-3a10abc3ab5c\") "
Oct 30 22:36:25.759428 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:25.759394    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-maas-api-openshift-ca" pod "80f6fd3f-54ba-482f-af68-3a10abc3ab5c" (UID: "80f6fd3f-54ba-482f-af68-3a10abc3ab5c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:36:25.761220 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:25.761147    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-maas-api-xks-ca" pod "80f6fd3f-54ba-482f-af68-3a10abc3ab5c" (UID: "80f6fd3f-54ba-482f-af68-3a10abc3ab5c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:36:25.761764 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:25.761740    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-kube-api-access-p9m2m" pod "80f6fd3f-54ba-482f-af68-3a10abc3ab5c" (UID: "80f6fd3f-54ba-482f-af68-3a10abc3ab5c"). InnerVolumeSpecName "kube-api-access-p9m2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:36:25.860533 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:25.860497    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:25.860533 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:25.860529    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:25.860533 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:25.860540    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9m2m\" (UniqueName: \"kubernetes.io/projected/80f6fd3f-54ba-482f-af68-3a10abc3ab5c-kube-api-access-p9m2m\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:26.520289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:26.519666    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm" event={"ID":"80f6fd3f-54ba-482f-af68-3a10abc3ab5c","Type":"ContainerDied","Data":"b4bb9fc5e20a7114ee6eeba9d94f911bed794a6a73001bb5c90cb9b3bb5b2fe2"}
Oct 30 22:36:26.520289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:26.519733    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4bb9fc5e20a7114ee6eeba9d94f911bed794a6a73001bb5c90cb9b3bb5b2fe2"
Oct 30 22:36:26.520289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:26.519802    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"
Oct 30 22:36:28.210443 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:28.210411    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:30.068482 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:30.068446    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"]
Oct 30 22:36:30.117464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:30.117434    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-573222d1-4nvmg"]
Oct 30 22:36:31.445883 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:31.445848    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df0ab2a5-f98a-4375-8b8c-3744fc903726" path="/var/lib/kubelet/pods/df0ab2a5-f98a-4375-8b8c-3744fc903726/volumes"
Oct 30 22:36:31.771896 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:31.771804    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:33.072198 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:33.072159    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:34.101629 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.101592    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:34.521287 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.521224    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"]
Oct 30 22:36:34.521687 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.521674    5586 cpu_manager.go:414] "containerMap: removing container" podUID="80f6fd3f-54ba-482f-af68-3a10abc3ab5c" containerName="revoke-keys"
Oct 30 22:36:34.521767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.521687    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80f6fd3f-54ba-482f-af68-3a10abc3ab5c" containerName="revoke-keys"
Oct 30 22:36:34.521825 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.521771    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="80f6fd3f-54ba-482f-af68-3a10abc3ab5c" containerName="revoke-keys"
Oct 30 22:36:34.525392 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.525370    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"
Oct 30 22:36:34.579551 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.579515    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:36:34.627562 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.627528    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:36:34.627752 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.627529    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:36:34.627752 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.627577    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:36:34.630130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.630102    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"]
Oct 30 22:36:34.647991 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.647956    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzsvw\" (UniqueName: \"kubernetes.io/projected/fa50ce37-4784-459b-b44c-9f988ac3881b-kube-api-access-tzsvw\") pod \"maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v\" (UID: \"fa50ce37-4784-459b-b44c-9f988ac3881b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"
Oct 30 22:36:34.647991 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.647995    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa50ce37-4784-459b-b44c-9f988ac3881b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v\" (UID: \"fa50ce37-4784-459b-b44c-9f988ac3881b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"
Oct 30 22:36:34.648247 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.648092    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa50ce37-4784-459b-b44c-9f988ac3881b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v\" (UID: \"fa50ce37-4784-459b-b44c-9f988ac3881b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"
Oct 30 22:36:34.749110 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.749077    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa50ce37-4784-459b-b44c-9f988ac3881b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v\" (UID: \"fa50ce37-4784-459b-b44c-9f988ac3881b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"
Oct 30 22:36:34.749313 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.749186    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzsvw\" (UniqueName: \"kubernetes.io/projected/fa50ce37-4784-459b-b44c-9f988ac3881b-kube-api-access-tzsvw\") pod \"maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v\" (UID: \"fa50ce37-4784-459b-b44c-9f988ac3881b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"
Oct 30 22:36:34.749313 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.749206    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa50ce37-4784-459b-b44c-9f988ac3881b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v\" (UID: \"fa50ce37-4784-459b-b44c-9f988ac3881b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"
Oct 30 22:36:34.749779 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.749751    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa50ce37-4784-459b-b44c-9f988ac3881b-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v\" (UID: \"fa50ce37-4784-459b-b44c-9f988ac3881b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"
Oct 30 22:36:34.751546 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.751523    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa50ce37-4784-459b-b44c-9f988ac3881b-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v\" (UID: \"fa50ce37-4784-459b-b44c-9f988ac3881b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"
Oct 30 22:36:34.806001 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.805921    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzsvw\" (UniqueName: \"kubernetes.io/projected/fa50ce37-4784-459b-b44c-9f988ac3881b-kube-api-access-tzsvw\") pod \"maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v\" (UID: \"fa50ce37-4784-459b-b44c-9f988ac3881b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"
Oct 30 22:36:34.839284 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:34.839253    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"
Oct 30 22:36:35.214430 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:35.214402    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"]
Oct 30 22:36:35.216482 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:36:35.216455    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa50ce37_4784_459b_b44c_9f988ac3881b.slice/crio-bdfb76de579f010132fdfd2ac4917520f52a88a615b5f460d7d113c8e05587af WatchSource:0}: Error finding container bdfb76de579f010132fdfd2ac4917520f52a88a615b5f460d7d113c8e05587af: Status 404 returned error can't find the container with id bdfb76de579f010132fdfd2ac4917520f52a88a615b5f460d7d113c8e05587af
Oct 30 22:36:35.579297 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:35.579251    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v" event={"ID":"fa50ce37-4784-459b-b44c-9f988ac3881b","Type":"ContainerStarted","Data":"d5d8ebf78391d99edce1e36253c9947950907ed4898b04e327b348e8c774bbb8"}
Oct 30 22:36:35.579485 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:35.579313    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v" event={"ID":"fa50ce37-4784-459b-b44c-9f988ac3881b","Type":"ContainerStarted","Data":"bdfb76de579f010132fdfd2ac4917520f52a88a615b5f460d7d113c8e05587af"}
Oct 30 22:36:35.695467 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:35.695411    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v" podStartSLOduration=1.695397109 podStartE2EDuration="1.695397109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:36:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:36:35.693389767 +0000 UTC m=+3738.365293701" watchObservedRunningTime="2026-10-30 22:36:35.695397109 +0000 UTC m=+3738.367301044"
Oct 30 22:36:36.587584 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:36.587548    5586 generic.go:356] "Generic (PLEG): container finished" podID="fa50ce37-4784-459b-b44c-9f988ac3881b" containerID="d5d8ebf78391d99edce1e36253c9947950907ed4898b04e327b348e8c774bbb8" exitCode=0
Oct 30 22:36:36.587978 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:36.587634    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v" event={"ID":"fa50ce37-4784-459b-b44c-9f988ac3881b","Type":"ContainerDied","Data":"d5d8ebf78391d99edce1e36253c9947950907ed4898b04e327b348e8c774bbb8"}
Oct 30 22:36:37.043438 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:37.043405    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:37.737952 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:37.737917    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"
Oct 30 22:36:37.882625 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:37.882519    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa50ce37-4784-459b-b44c-9f988ac3881b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa50ce37-4784-459b-b44c-9f988ac3881b-maas-api-xks-ca\") pod \"fa50ce37-4784-459b-b44c-9f988ac3881b\" (UID: \"fa50ce37-4784-459b-b44c-9f988ac3881b\") "
Oct 30 22:36:37.882625 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:37.882575    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa50ce37-4784-459b-b44c-9f988ac3881b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa50ce37-4784-459b-b44c-9f988ac3881b-maas-api-openshift-ca\") pod \"fa50ce37-4784-459b-b44c-9f988ac3881b\" (UID: \"fa50ce37-4784-459b-b44c-9f988ac3881b\") "
Oct 30 22:36:37.882914 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:37.882663    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa50ce37-4784-459b-b44c-9f988ac3881b-kube-api-access-tzsvw\" (UniqueName: \"kubernetes.io/projected/fa50ce37-4784-459b-b44c-9f988ac3881b-kube-api-access-tzsvw\") pod \"fa50ce37-4784-459b-b44c-9f988ac3881b\" (UID: \"fa50ce37-4784-459b-b44c-9f988ac3881b\") "
Oct 30 22:36:37.883033 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:37.883007    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa50ce37-4784-459b-b44c-9f988ac3881b-maas-api-openshift-ca" pod "fa50ce37-4784-459b-b44c-9f988ac3881b" (UID: "fa50ce37-4784-459b-b44c-9f988ac3881b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:36:37.884980 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:37.884941    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa50ce37-4784-459b-b44c-9f988ac3881b-kube-api-access-tzsvw" pod "fa50ce37-4784-459b-b44c-9f988ac3881b" (UID: "fa50ce37-4784-459b-b44c-9f988ac3881b"). InnerVolumeSpecName "kube-api-access-tzsvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:36:37.885326 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:37.885303    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa50ce37-4784-459b-b44c-9f988ac3881b-maas-api-xks-ca" pod "fa50ce37-4784-459b-b44c-9f988ac3881b" (UID: "fa50ce37-4784-459b-b44c-9f988ac3881b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:36:37.984003 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:37.983965    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/fa50ce37-4784-459b-b44c-9f988ac3881b-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:37.984003 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:37.983998    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/fa50ce37-4784-459b-b44c-9f988ac3881b-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:37.984003 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:37.984009    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzsvw\" (UniqueName: \"kubernetes.io/projected/fa50ce37-4784-459b-b44c-9f988ac3881b-kube-api-access-tzsvw\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:38.192624 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:38.192587    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:38.600931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:38.600847    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"
Oct 30 22:36:38.600931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:38.600875    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v" event={"ID":"fa50ce37-4784-459b-b44c-9f988ac3881b","Type":"ContainerDied","Data":"bdfb76de579f010132fdfd2ac4917520f52a88a615b5f460d7d113c8e05587af"}
Oct 30 22:36:38.600931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:38.600927    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdfb76de579f010132fdfd2ac4917520f52a88a615b5f460d7d113c8e05587af"
Oct 30 22:36:39.086481 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:39.086444    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:43.002403 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:43.002366    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:45.360009 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:45.359963    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:45.577830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:45.577791    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:47.692162 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:47.692107    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:48.360859 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:36:48.360815    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:36:51.877788 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:51.877748    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:53.477077 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:53.477036    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:53.771363 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:53.771270    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:54.069372 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:54.069286    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"]
Oct 30 22:36:54.118836 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:54.118798    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-aafbca-7st6v"]
Oct 30 22:36:54.562109 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:54.562065    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:55.453907 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:36:55.453866    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa50ce37-4784-459b-b44c-9f988ac3881b" path="/var/lib/kubelet/pods/fa50ce37-4784-459b-b44c-9f988ac3881b/volumes"
Oct 30 22:37:00.448911 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:37:00.448877    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:00.607923 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:37:00.607879    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:00.816609 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:37:00.816508    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:07.801669 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:37:07.801630    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:12.051049 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:37:12.051004    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"]
Oct 30 22:37:12.098222 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:37:12.098184    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-pxn9p"]
Oct 30 22:37:13.450692 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:37:13.450649    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5822a27-d455-4f22-875c-7483c087c8c9" path="/var/lib/kubelet/pods/a5822a27-d455-4f22-875c-7483c087c8c9/volumes"
Oct 30 22:37:15.415210 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:37:15.415174    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:16.074780 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:37:16.074727    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:18.365888 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:37:18.365843    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:37:31.019556 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:37:31.019461    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:37.407579 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:37:37.407539    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:48.361589 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:37:48.361549    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:37:50.389745 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:37:50.389710    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:50.802321 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:37:50.802289    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:51.590308 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:37:51.590263    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:18.361625 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:38:18.361583    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:38:21.700597 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:21.700554    5586 scope.go:122] "RemoveContainer" containerID="fe9e34d58d06ee4f6adb72772bf71ac04eed236b2be1a21050e19d8f840cad23"
Oct 30 22:38:21.732623 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:21.732471    5586 scope.go:122] "RemoveContainer" containerID="7396d72e8eba49193c261915f6bd7df3da1fd17c0207397f77e4c93fae5f1c25"
Oct 30 22:38:21.747275 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:21.747204    5586 scope.go:122] "RemoveContainer" containerID="3eaa42d13c44676a757f7185e6ed0774a72f4815906738baefe225b69b8d98f5"
Oct 30 22:38:29.824327 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:29.824280    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"]
Oct 30 22:38:29.824968 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:29.824947    5586 cpu_manager.go:414] "containerMap: removing container" podUID="fa50ce37-4784-459b-b44c-9f988ac3881b" containerName="revoke-keys"
Oct 30 22:38:29.825014 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:29.824972    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa50ce37-4784-459b-b44c-9f988ac3881b" containerName="revoke-keys"
Oct 30 22:38:29.825078 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:29.825068    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa50ce37-4784-459b-b44c-9f988ac3881b" containerName="revoke-keys"
Oct 30 22:38:29.828831 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:29.828808    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"
Oct 30 22:38:29.884146 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:29.884110    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"]
Oct 30 22:38:29.930525 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:29.930485    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:38:29.930719 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:29.930490    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:38:29.930719 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:29.930498    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:38:29.930719 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:29.930536    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:38:30.006565 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:30.006526    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j7sp\" (UniqueName: \"kubernetes.io/projected/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-kube-api-access-7j7sp\") pod \"maas-api-revoke-sub-452b6d7b-p879s\" (UID: \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"
Oct 30 22:38:30.006767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:30.006649    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-p879s\" (UID: \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"
Oct 30 22:38:30.006767 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:30.006738    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-p879s\" (UID: \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"
Oct 30 22:38:30.107837 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:30.107745    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-p879s\" (UID: \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"
Oct 30 22:38:30.108003 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:30.107833    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j7sp\" (UniqueName: \"kubernetes.io/projected/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-kube-api-access-7j7sp\") pod \"maas-api-revoke-sub-452b6d7b-p879s\" (UID: \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"
Oct 30 22:38:30.108003 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:30.107915    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-p879s\" (UID: \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"
Oct 30 22:38:30.108702 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:30.108672    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-p879s\" (UID: \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"
Oct 30 22:38:30.110181 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:30.110150    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-p879s\" (UID: \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"
Oct 30 22:38:30.166174 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:30.166138    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j7sp\" (UniqueName: \"kubernetes.io/projected/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-kube-api-access-7j7sp\") pod \"maas-api-revoke-sub-452b6d7b-p879s\" (UID: \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"
Oct 30 22:38:30.445922 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:30.445878    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"
Oct 30 22:38:30.819266 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:30.819222    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"]
Oct 30 22:38:30.822025 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:38:30.822004    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a0500b4_46f7_44e9_bd74_0f25ac717f4c.slice/crio-e0d262297edd01bb5a1118ad6960833c5fc19eef27b56ecd95b484a83a9f774a WatchSource:0}: Error finding container e0d262297edd01bb5a1118ad6960833c5fc19eef27b56ecd95b484a83a9f774a: Status 404 returned error can't find the container with id e0d262297edd01bb5a1118ad6960833c5fc19eef27b56ecd95b484a83a9f774a
Oct 30 22:38:31.408600 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:31.408559    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s" event={"ID":"0a0500b4-46f7-44e9-bd74-0f25ac717f4c","Type":"ContainerStarted","Data":"df2f1b6d2848f7ca8e387fb6d3c05877caeaae39fe708eabf07a634d17a1b506"}
Oct 30 22:38:31.408997 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:31.408625    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s" event={"ID":"0a0500b4-46f7-44e9-bd74-0f25ac717f4c","Type":"ContainerStarted","Data":"e0d262297edd01bb5a1118ad6960833c5fc19eef27b56ecd95b484a83a9f774a"}
Oct 30 22:38:31.526534 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:31.526480    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s" podStartSLOduration=2.52646184 podStartE2EDuration="2.52646184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:38:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:38:31.523728334 +0000 UTC m=+3854.195632270" watchObservedRunningTime="2026-10-30 22:38:31.52646184 +0000 UTC m=+3854.198365775"
Oct 30 22:38:32.416394 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:32.416358    5586 generic.go:356] "Generic (PLEG): container finished" podID="0a0500b4-46f7-44e9-bd74-0f25ac717f4c" containerID="df2f1b6d2848f7ca8e387fb6d3c05877caeaae39fe708eabf07a634d17a1b506" exitCode=0
Oct 30 22:38:32.416794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:32.416441    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s" event={"ID":"0a0500b4-46f7-44e9-bd74-0f25ac717f4c","Type":"ContainerDied","Data":"df2f1b6d2848f7ca8e387fb6d3c05877caeaae39fe708eabf07a634d17a1b506"}
Oct 30 22:38:33.569576 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:33.569546    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"
Oct 30 22:38:33.640895 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:33.640859    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-kube-api-access-7j7sp\" (UniqueName: \"kubernetes.io/projected/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-kube-api-access-7j7sp\") pod \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\" (UID: \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\") "
Oct 30 22:38:33.641083 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:33.640915    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-maas-api-xks-ca\") pod \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\" (UID: \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\") "
Oct 30 22:38:33.641083 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:33.641014    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-maas-api-openshift-ca\") pod \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\" (UID: \"0a0500b4-46f7-44e9-bd74-0f25ac717f4c\") "
Oct 30 22:38:33.641417 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:33.641396    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-maas-api-openshift-ca" pod "0a0500b4-46f7-44e9-bd74-0f25ac717f4c" (UID: "0a0500b4-46f7-44e9-bd74-0f25ac717f4c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:38:33.643106 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:33.643083    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-kube-api-access-7j7sp" pod "0a0500b4-46f7-44e9-bd74-0f25ac717f4c" (UID: "0a0500b4-46f7-44e9-bd74-0f25ac717f4c"). InnerVolumeSpecName "kube-api-access-7j7sp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:38:33.643106 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:33.643095    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-maas-api-xks-ca" pod "0a0500b4-46f7-44e9-bd74-0f25ac717f4c" (UID: "0a0500b4-46f7-44e9-bd74-0f25ac717f4c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:38:33.741836 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:33.741744    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j7sp\" (UniqueName: \"kubernetes.io/projected/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-kube-api-access-7j7sp\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:33.741836 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:33.741780    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:33.741836 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:33.741790    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0a0500b4-46f7-44e9-bd74-0f25ac717f4c-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:34.430343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:34.430303    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s" event={"ID":"0a0500b4-46f7-44e9-bd74-0f25ac717f4c","Type":"ContainerDied","Data":"e0d262297edd01bb5a1118ad6960833c5fc19eef27b56ecd95b484a83a9f774a"}
Oct 30 22:38:34.430538 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:34.430358    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0d262297edd01bb5a1118ad6960833c5fc19eef27b56ecd95b484a83a9f774a"
Oct 30 22:38:34.430538 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:34.430319    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"
Oct 30 22:38:36.658185 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:36.658154    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:37.737615 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:37.737564    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:39.187165 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:39.187121    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:42.646146 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:42.646110    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:48.493564 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:38:48.493499    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:38:57.096540 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:38:57.096495    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:09.930341 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:09.930207    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:11.366242 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.366184    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"]
Oct 30 22:39:11.366899 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.366876    5586 cpu_manager.go:414] "containerMap: removing container" podUID="0a0500b4-46f7-44e9-bd74-0f25ac717f4c" containerName="revoke-keys"
Oct 30 22:39:11.366998 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.366901    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a0500b4-46f7-44e9-bd74-0f25ac717f4c" containerName="revoke-keys"
Oct 30 22:39:11.367060 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.367012    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a0500b4-46f7-44e9-bd74-0f25ac717f4c" containerName="revoke-keys"
Oct 30 22:39:11.369201 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.369179    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"
Oct 30 22:39:11.403085 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.403040    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh8w2\" (UniqueName: \"kubernetes.io/projected/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-kube-api-access-mh8w2\") pod \"maas-api-revoke-sub-3e31a506-2754h\" (UID: \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"
Oct 30 22:39:11.403286 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.403144    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-2754h\" (UID: \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"
Oct 30 22:39:11.403286 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.403200    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-2754h\" (UID: \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"
Oct 30 22:39:11.427135 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.427106    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:39:11.472318 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.472282    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:39:11.472318 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.472280    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:39:11.472582 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.472348    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:39:11.474922 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.474888    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"]
Oct 30 22:39:11.503786 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.503752    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-2754h\" (UID: \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"
Oct 30 22:39:11.503990 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.503820    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh8w2\" (UniqueName: \"kubernetes.io/projected/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-kube-api-access-mh8w2\") pod \"maas-api-revoke-sub-3e31a506-2754h\" (UID: \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"
Oct 30 22:39:11.503990 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.503861    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-2754h\" (UID: \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"
Oct 30 22:39:11.504639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.504610    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-2754h\" (UID: \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"
Oct 30 22:39:11.506187 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.506164    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-2754h\" (UID: \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"
Oct 30 22:39:11.566977 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.566931    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh8w2\" (UniqueName: \"kubernetes.io/projected/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-kube-api-access-mh8w2\") pod \"maas-api-revoke-sub-3e31a506-2754h\" (UID: \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"
Oct 30 22:39:11.681125 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:11.681079    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"
Oct 30 22:39:12.117405 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:12.117315    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"]
Oct 30 22:39:12.134419 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:39:12.121972    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17f1988a_2a8b_4cb4_b4d5_d2c5138c1e96.slice/crio-4813b932d0ece326bacaa5ce7d0f11ba738b3a0b5ab068fef2c73faa535749cf WatchSource:0}: Error finding container 4813b932d0ece326bacaa5ce7d0f11ba738b3a0b5ab068fef2c73faa535749cf: Status 404 returned error can't find the container with id 4813b932d0ece326bacaa5ce7d0f11ba738b3a0b5ab068fef2c73faa535749cf
Oct 30 22:39:12.771269 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:12.768497    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h" event={"ID":"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96","Type":"ContainerStarted","Data":"680e7b760bdf56812741bf8a45387a440d13c89de07f2666ed54a7212d66e88b"}
Oct 30 22:39:12.771269 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:12.768566    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h" event={"ID":"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96","Type":"ContainerStarted","Data":"4813b932d0ece326bacaa5ce7d0f11ba738b3a0b5ab068fef2c73faa535749cf"}
Oct 30 22:39:12.886310 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:12.886011    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h" podStartSLOduration=1.885990591 podStartE2EDuration="1.885990591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:39:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:39:12.884687061 +0000 UTC m=+3895.556590997" watchObservedRunningTime="2026-10-30 22:39:12.885990591 +0000 UTC m=+3895.557894527"
Oct 30 22:39:13.794259 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:13.794133    5586 generic.go:356] "Generic (PLEG): container finished" podID="17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96" containerID="680e7b760bdf56812741bf8a45387a440d13c89de07f2666ed54a7212d66e88b" exitCode=0
Oct 30 22:39:13.794794 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:13.794303    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h" event={"ID":"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96","Type":"ContainerDied","Data":"680e7b760bdf56812741bf8a45387a440d13c89de07f2666ed54a7212d66e88b"}
Oct 30 22:39:14.944157 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:14.944129    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"
Oct 30 22:39:15.112182 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:15.112076    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-kube-api-access-mh8w2\" (UniqueName: \"kubernetes.io/projected/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-kube-api-access-mh8w2\") pod \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\" (UID: \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\") "
Oct 30 22:39:15.112182 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:15.112141    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-maas-api-openshift-ca\") pod \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\" (UID: \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\") "
Oct 30 22:39:15.112471 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:15.112185    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-maas-api-xks-ca\") pod \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\" (UID: \"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96\") "
Oct 30 22:39:15.112630 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:15.112593    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-maas-api-openshift-ca" pod "17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96" (UID: "17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:39:15.114299 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:15.114270    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-kube-api-access-mh8w2" pod "17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96" (UID: "17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96"). InnerVolumeSpecName "kube-api-access-mh8w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:39:15.114423 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:15.114408    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-maas-api-xks-ca" pod "17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96" (UID: "17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:39:15.213838 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:15.213800    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mh8w2\" (UniqueName: \"kubernetes.io/projected/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-kube-api-access-mh8w2\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:39:15.213838 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:15.213832    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:39:15.213838 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:15.213843    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:39:15.807422 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:15.807338    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"
Oct 30 22:39:15.807579 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:15.807339    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h" event={"ID":"17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96","Type":"ContainerDied","Data":"4813b932d0ece326bacaa5ce7d0f11ba738b3a0b5ab068fef2c73faa535749cf"}
Oct 30 22:39:15.807579 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:15.807465    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4813b932d0ece326bacaa5ce7d0f11ba738b3a0b5ab068fef2c73faa535749cf"
Oct 30 22:39:16.046118 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:16.046076    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"]
Oct 30 22:39:16.048268 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:16.048225    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-wklxv"]
Oct 30 22:39:17.447301 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:17.447266    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9494a779-aacb-4dd1-9e78-0061a552799d" path="/var/lib/kubelet/pods/9494a779-aacb-4dd1-9e78-0061a552799d/volumes"
Oct 30 22:39:18.214602 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:18.214568    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:39:18.219971 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:18.219941    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:39:18.226130 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:18.226106    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:39:18.230832 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:18.230811    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:39:18.360490 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:39:18.360453    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:39:38.070888 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:38.070847    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:45.056644 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:45.056605    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"]
Oct 30 22:39:45.105846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:45.105807    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-d7689"]
Oct 30 22:39:45.446171 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:45.446138    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ede83587-9e6f-486a-ab06-d7d87f985788" path="/var/lib/kubelet/pods/ede83587-9e6f-486a-ab06-d7d87f985788/volumes"
Oct 30 22:39:48.361078 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:39:48.361032    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:39:51.982993 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:51.982958    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"]
Oct 30 22:39:51.983435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:51.983420    5586 cpu_manager.go:414] "containerMap: removing container" podUID="17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96" containerName="revoke-keys"
Oct 30 22:39:51.983484 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:51.983436    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96" containerName="revoke-keys"
Oct 30 22:39:51.983521 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:51.983511    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96" containerName="revoke-keys"
Oct 30 22:39:51.986943 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:51.986927    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"
Oct 30 22:39:52.045186 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.045151    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"]
Oct 30 22:39:52.056770 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.056743    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5428150b-ed70-4c23-bf07-56b07d376863-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2b53613-xq85k\" (UID: \"5428150b-ed70-4c23-bf07-56b07d376863\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"
Oct 30 22:39:52.056934 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.056811    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5428150b-ed70-4c23-bf07-56b07d376863-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2b53613-xq85k\" (UID: \"5428150b-ed70-4c23-bf07-56b07d376863\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"
Oct 30 22:39:52.056985 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.056955    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6wjc\" (UniqueName: \"kubernetes.io/projected/5428150b-ed70-4c23-bf07-56b07d376863-kube-api-access-x6wjc\") pod \"maas-api-revoke-sub-e2b53613-xq85k\" (UID: \"5428150b-ed70-4c23-bf07-56b07d376863\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"
Oct 30 22:39:52.091340 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.091302    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:39:52.091340 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.091319    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:39:52.091596 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.091432    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lnxvw\""
Oct 30 22:39:52.091596 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.091529    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:39:52.158292 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.158257    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5428150b-ed70-4c23-bf07-56b07d376863-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2b53613-xq85k\" (UID: \"5428150b-ed70-4c23-bf07-56b07d376863\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"
Oct 30 22:39:52.158498 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.158337    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6wjc\" (UniqueName: \"kubernetes.io/projected/5428150b-ed70-4c23-bf07-56b07d376863-kube-api-access-x6wjc\") pod \"maas-api-revoke-sub-e2b53613-xq85k\" (UID: \"5428150b-ed70-4c23-bf07-56b07d376863\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"
Oct 30 22:39:52.158498 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.158417    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5428150b-ed70-4c23-bf07-56b07d376863-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2b53613-xq85k\" (UID: \"5428150b-ed70-4c23-bf07-56b07d376863\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"
Oct 30 22:39:52.158994 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.158973    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5428150b-ed70-4c23-bf07-56b07d376863-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2b53613-xq85k\" (UID: \"5428150b-ed70-4c23-bf07-56b07d376863\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"
Oct 30 22:39:52.160799 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.160779    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5428150b-ed70-4c23-bf07-56b07d376863-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2b53613-xq85k\" (UID: \"5428150b-ed70-4c23-bf07-56b07d376863\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"
Oct 30 22:39:52.215103 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.215069    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6wjc\" (UniqueName: \"kubernetes.io/projected/5428150b-ed70-4c23-bf07-56b07d376863-kube-api-access-x6wjc\") pod \"maas-api-revoke-sub-e2b53613-xq85k\" (UID: \"5428150b-ed70-4c23-bf07-56b07d376863\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"
Oct 30 22:39:52.299405 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.299318    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"
Oct 30 22:39:52.673909 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:52.673703    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"]
Oct 30 22:39:52.676780 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:39:52.676751    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5428150b_ed70_4c23_bf07_56b07d376863.slice/crio-d20560ea8b0113d97e95824ca7103e233ef1785deb33778067f54de4cf08d50a WatchSource:0}: Error finding container d20560ea8b0113d97e95824ca7103e233ef1785deb33778067f54de4cf08d50a: Status 404 returned error can't find the container with id d20560ea8b0113d97e95824ca7103e233ef1785deb33778067f54de4cf08d50a
Oct 30 22:39:53.035285 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:53.035212    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k" event={"ID":"5428150b-ed70-4c23-bf07-56b07d376863","Type":"ContainerStarted","Data":"1e35a4cfc0e4635550bfa1186b55d482fd280853585ed86a7ee02a202f9f9bde"}
Oct 30 22:39:53.035285 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:53.035291    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k" event={"ID":"5428150b-ed70-4c23-bf07-56b07d376863","Type":"ContainerStarted","Data":"d20560ea8b0113d97e95824ca7103e233ef1785deb33778067f54de4cf08d50a"}
Oct 30 22:39:53.156448 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:53.156388    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k" podStartSLOduration=2.156372287 podStartE2EDuration="2.156372287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:39:53.155143391 +0000 UTC m=+3935.827047327" watchObservedRunningTime="2026-10-30 22:39:53.156372287 +0000 UTC m=+3935.828276218"
Oct 30 22:39:54.042559 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:54.042518    5586 generic.go:356] "Generic (PLEG): container finished" podID="5428150b-ed70-4c23-bf07-56b07d376863" containerID="1e35a4cfc0e4635550bfa1186b55d482fd280853585ed86a7ee02a202f9f9bde" exitCode=0
Oct 30 22:39:54.042979 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:54.042600    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k" event={"ID":"5428150b-ed70-4c23-bf07-56b07d376863","Type":"ContainerDied","Data":"1e35a4cfc0e4635550bfa1186b55d482fd280853585ed86a7ee02a202f9f9bde"}
Oct 30 22:39:55.189630 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:55.189597    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"
Oct 30 22:39:55.287759 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:55.287725    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5428150b-ed70-4c23-bf07-56b07d376863-kube-api-access-x6wjc\" (UniqueName: \"kubernetes.io/projected/5428150b-ed70-4c23-bf07-56b07d376863-kube-api-access-x6wjc\") pod \"5428150b-ed70-4c23-bf07-56b07d376863\" (UID: \"5428150b-ed70-4c23-bf07-56b07d376863\") "
Oct 30 22:39:55.287978 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:55.287799    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5428150b-ed70-4c23-bf07-56b07d376863-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5428150b-ed70-4c23-bf07-56b07d376863-maas-api-xks-ca\") pod \"5428150b-ed70-4c23-bf07-56b07d376863\" (UID: \"5428150b-ed70-4c23-bf07-56b07d376863\") "
Oct 30 22:39:55.287978 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:55.287915    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5428150b-ed70-4c23-bf07-56b07d376863-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5428150b-ed70-4c23-bf07-56b07d376863-maas-api-openshift-ca\") pod \"5428150b-ed70-4c23-bf07-56b07d376863\" (UID: \"5428150b-ed70-4c23-bf07-56b07d376863\") "
Oct 30 22:39:55.288390 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:55.288359    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5428150b-ed70-4c23-bf07-56b07d376863-maas-api-openshift-ca" pod "5428150b-ed70-4c23-bf07-56b07d376863" (UID: "5428150b-ed70-4c23-bf07-56b07d376863"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:39:55.290080 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:55.290044    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5428150b-ed70-4c23-bf07-56b07d376863-maas-api-xks-ca" pod "5428150b-ed70-4c23-bf07-56b07d376863" (UID: "5428150b-ed70-4c23-bf07-56b07d376863"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:39:55.290218 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:55.290130    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5428150b-ed70-4c23-bf07-56b07d376863-kube-api-access-x6wjc" pod "5428150b-ed70-4c23-bf07-56b07d376863" (UID: "5428150b-ed70-4c23-bf07-56b07d376863"). InnerVolumeSpecName "kube-api-access-x6wjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:39:55.389152 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:55.389050    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6wjc\" (UniqueName: \"kubernetes.io/projected/5428150b-ed70-4c23-bf07-56b07d376863-kube-api-access-x6wjc\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:39:55.389152 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:55.389097    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5428150b-ed70-4c23-bf07-56b07d376863-maas-api-xks-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:39:55.389152 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:55.389112    5586 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5428150b-ed70-4c23-bf07-56b07d376863-maas-api-openshift-ca\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:39:56.056464 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:56.056433    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"
Oct 30 22:39:56.056658 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:56.056429    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k" event={"ID":"5428150b-ed70-4c23-bf07-56b07d376863","Type":"ContainerDied","Data":"d20560ea8b0113d97e95824ca7103e233ef1785deb33778067f54de4cf08d50a"}
Oct 30 22:39:56.056658 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:39:56.056551    5586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d20560ea8b0113d97e95824ca7103e233ef1785deb33778067f54de4cf08d50a"
Oct 30 22:40:11.898187 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:11.898142    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:12.507044 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:12.506993    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:12.689449 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:12.689414    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:13.548341 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.548301    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"]
Oct 30 22:40:13.548890 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.548765    5586 cpu_manager.go:414] "containerMap: removing container" podUID="5428150b-ed70-4c23-bf07-56b07d376863" containerName="revoke-keys"
Oct 30 22:40:13.548890 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.548777    5586 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5428150b-ed70-4c23-bf07-56b07d376863" containerName="revoke-keys"
Oct 30 22:40:13.548890 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.548864    5586 memory_manager.go:365] "RemoveStaleState removing state" podUID="5428150b-ed70-4c23-bf07-56b07d376863" containerName="revoke-keys"
Oct 30 22:40:13.552214 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.552194    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:13.653991 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.653954    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 22:40:13.654167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.654044    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 22:40:13.654167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.654157    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 22:40:13.654317 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.654196    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 22:40:13.654317 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.654218    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fqgfc\""
Oct 30 22:40:13.656723 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.656704    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"]
Oct 30 22:40:13.678045 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.678016    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9f770df-e0dd-42bd-b5e3-b777bfd20691-openshift-service-ca\") pod \"maas-api-54959df7f7-w7d92\" (UID: \"c9f770df-e0dd-42bd-b5e3-b777bfd20691\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:13.678197 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.678052    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhrxz\" (UniqueName: \"kubernetes.io/projected/c9f770df-e0dd-42bd-b5e3-b777bfd20691-kube-api-access-dhrxz\") pod \"maas-api-54959df7f7-w7d92\" (UID: \"c9f770df-e0dd-42bd-b5e3-b777bfd20691\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:13.678197 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.678185    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9f770df-e0dd-42bd-b5e3-b777bfd20691-maas-api-tls\") pod \"maas-api-54959df7f7-w7d92\" (UID: \"c9f770df-e0dd-42bd-b5e3-b777bfd20691\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:13.678306 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.678209    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9f770df-e0dd-42bd-b5e3-b777bfd20691-metrics-certs\") pod \"maas-api-54959df7f7-w7d92\" (UID: \"c9f770df-e0dd-42bd-b5e3-b777bfd20691\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:13.779219 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.779185    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9f770df-e0dd-42bd-b5e3-b777bfd20691-maas-api-tls\") pod \"maas-api-54959df7f7-w7d92\" (UID: \"c9f770df-e0dd-42bd-b5e3-b777bfd20691\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:13.779453 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.779225    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9f770df-e0dd-42bd-b5e3-b777bfd20691-metrics-certs\") pod \"maas-api-54959df7f7-w7d92\" (UID: \"c9f770df-e0dd-42bd-b5e3-b777bfd20691\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:13.779453 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.779274    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9f770df-e0dd-42bd-b5e3-b777bfd20691-openshift-service-ca\") pod \"maas-api-54959df7f7-w7d92\" (UID: \"c9f770df-e0dd-42bd-b5e3-b777bfd20691\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:13.779453 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.779296    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhrxz\" (UniqueName: \"kubernetes.io/projected/c9f770df-e0dd-42bd-b5e3-b777bfd20691-kube-api-access-dhrxz\") pod \"maas-api-54959df7f7-w7d92\" (UID: \"c9f770df-e0dd-42bd-b5e3-b777bfd20691\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:13.780081 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.780056    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9f770df-e0dd-42bd-b5e3-b777bfd20691-openshift-service-ca\") pod \"maas-api-54959df7f7-w7d92\" (UID: \"c9f770df-e0dd-42bd-b5e3-b777bfd20691\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:13.781906 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.781879    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9f770df-e0dd-42bd-b5e3-b777bfd20691-maas-api-tls\") pod \"maas-api-54959df7f7-w7d92\" (UID: \"c9f770df-e0dd-42bd-b5e3-b777bfd20691\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:13.782008 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.781879    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9f770df-e0dd-42bd-b5e3-b777bfd20691-metrics-certs\") pod \"maas-api-54959df7f7-w7d92\" (UID: \"c9f770df-e0dd-42bd-b5e3-b777bfd20691\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:13.835941 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.835862    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhrxz\" (UniqueName: \"kubernetes.io/projected/c9f770df-e0dd-42bd-b5e3-b777bfd20691-kube-api-access-dhrxz\") pod \"maas-api-54959df7f7-w7d92\" (UID: \"c9f770df-e0dd-42bd-b5e3-b777bfd20691\") " pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:13.865966 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:13.865919    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:14.260858 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:14.260830    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"]
Oct 30 22:40:14.262919 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:40:14.262881    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9f770df_e0dd_42bd_b5e3_b777bfd20691.slice/crio-acf1cfc68bdfc587b4ad27e195dcbbd16d87289bd76892d1c00255f3b897d4ff WatchSource:0}: Error finding container acf1cfc68bdfc587b4ad27e195dcbbd16d87289bd76892d1c00255f3b897d4ff: Status 404 returned error can't find the container with id acf1cfc68bdfc587b4ad27e195dcbbd16d87289bd76892d1c00255f3b897d4ff
Oct 30 22:40:14.264261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:14.264225    5586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:40:15.171286 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:15.171224    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92" event={"ID":"c9f770df-e0dd-42bd-b5e3-b777bfd20691","Type":"ContainerStarted","Data":"acf1cfc68bdfc587b4ad27e195dcbbd16d87289bd76892d1c00255f3b897d4ff"}
Oct 30 22:40:15.866613 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:15.866576    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:16.086654 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:16.086617    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:16.180008 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:16.179941    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92" event={"ID":"c9f770df-e0dd-42bd-b5e3-b777bfd20691","Type":"ContainerStarted","Data":"65ff50859e84218353f5f8971f5e3978bc621e51eaf787a806d0c8b0965e1e38"}
Oct 30 22:40:16.180584 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:16.180026    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:16.299796 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:16.299736    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92" podStartSLOduration=2.402421046 podStartE2EDuration="3.299722697s" totalImagesPullingTime="897.301651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:40:16.2979253 +0000 UTC m=+3958.969829237" watchObservedRunningTime="2026-10-30 22:40:16.299722697 +0000 UTC m=+3958.971626632"
Oct 30 22:40:18.361760 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:40:18.361708    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:40:19.110717 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:19.110685    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:19.428216 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:19.428178    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:20.526621 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:20.526582    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:20.706833 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:20.706794    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:20.861141 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:20.861047    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:21.800748 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:21.800714    5586 scope.go:122] "RemoveContainer" containerID="c2fc9c8868b926ea6cf099877aed3901af9631aa46298cc6ec0e619aafb80115"
Oct 30 22:40:22.193828 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:22.193796    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-54959df7f7-w7d92"
Oct 30 22:40:25.989849 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:25.989813    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:30.966406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:30.966369    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:35.778167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:35.778119    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:38.404024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:38.403984    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:40.686918 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:40.686878    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:40.843846 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:40.843810    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:40.995430 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:40.995339    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:45.703352 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:45.703315    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:48.361662 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:40:48.361616    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:40:50.479822 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:50.479783    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:55.512780 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:40:55.512733    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:00.303603 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:00.303570    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:02.374451 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:02.374407    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:05.079167 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:05.079109    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:09.897617 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:09.897572    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:13.040444 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:13.040403    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:13.280288 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:13.280228    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:14.684935 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:14.684888    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:18.363058 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:41:18.363016    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:41:18.920433 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:18.920396    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:20.445438 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:20.445392    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:20.725005 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:20.724923    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:21.824379 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:21.824346    5586 scope.go:122] "RemoveContainer" containerID="946a1cd2eb61e8e3ed03526d4f0c12c33b5a7bdda07a97038e83f422e48631eb"
Oct 30 22:41:23.411633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:23.411587    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:26.046625 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:26.046584    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"]
Oct 30 22:41:26.095784 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:26.095746    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e1e6bd7b-j57zm"]
Oct 30 22:41:27.452512 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:27.452465    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80f6fd3f-54ba-482f-af68-3a10abc3ab5c" path="/var/lib/kubelet/pods/80f6fd3f-54ba-482f-af68-3a10abc3ab5c/volumes"
Oct 30 22:41:34.648226 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:34.648181    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:35.389292 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:35.389220    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:35.568057 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:35.568019    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:38.733591 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:38.733548    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:39.793178 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:39.793134    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:40.000687 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:40.000651    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:42.266487 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:42.266437    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:43.056903 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:43.056865    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:43.235553 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:43.235516    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:46.039163 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:46.039125    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:46.195757 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:46.195720    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:46.366156 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:46.366071    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:48.361631 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:41:48.361594    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:41:50.996151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:50.996114    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:55.874624 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:41:55.874580    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:00.709631 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:00.709529    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:04.648110 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:04.648070    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:05.871180 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:05.871138    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:10.699538 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:10.699500    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:15.517685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:15.517652    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:18.361728 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:42:18.361685    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:42:20.288134 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:20.288095    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:25.136986 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:25.136948    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:30.088544 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:30.088501    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:34.988931 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:34.988883    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:39.072062 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:39.072015    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:39.895721 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:39.895682    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:44.722343 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:44.722300    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:48.362537 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:42:48.362497    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:42:49.590838 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:49.590797    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:54.479284 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:54.479217    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:59.286895 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:42:59.286854    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:04.080783 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:04.080730    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:08.795703 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:08.795666    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:13.557783 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:13.557750    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:18.360957 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:43:18.360918    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:43:18.386163 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:18.386122    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:21.767435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:21.767386    5586 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal" podUID="6ea3e08f52bca920602c76ad35bcaf69" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Oct 30 22:43:21.856461 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:21.856430    5586 scope.go:122] "RemoveContainer" containerID="d5d8ebf78391d99edce1e36253c9947950907ed4898b04e327b348e8c774bbb8"
Oct 30 22:43:21.865991 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:21.865968    5586 scope.go:122] "RemoveContainer" containerID="4a1473931c3270bd1a168e19f895c9d0740eee70d750f17dfc4817cd451ac9f1"
Oct 30 22:43:23.295916 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:23.295862    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:28.531288 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:28.531159    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:33.054042 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:33.053998    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"]
Oct 30 22:43:33.103663 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:33.103622    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-p879s"]
Oct 30 22:43:33.447106 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:33.447063    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a0500b4-46f7-44e9-bd74-0f25ac717f4c" path="/var/lib/kubelet/pods/0a0500b4-46f7-44e9-bd74-0f25ac717f4c/volumes"
Oct 30 22:43:33.485723 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:33.485692    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:35.960587 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:35.960548    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:36.172008 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:36.171977    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:38.303476 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:38.303442    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:39.427134 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:39.427092    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:47.692534 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:47.692499    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:48.362532 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:43:48.362491    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:43:49.285316 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:49.285283    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:49.622044 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:49.621926    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:50.374960 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:50.374922    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:56.610918 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:56.610878    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:56.755960 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:56.755927    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:56.973393 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:43:56.973348    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:15.062136 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:44:15.062089    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"]
Oct 30 22:44:15.111685 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:44:15.111651    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-2754h"]
Oct 30 22:44:15.446405 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:44:15.446365    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96" path="/var/lib/kubelet/pods/17f1988a-2a8b-4cb4-b4d5-d2c5138c1e96/volumes"
Oct 30 22:44:18.382326 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:44:18.382285    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:44:18.406024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:44:18.405991    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:44:18.434133 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:44:18.434091    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:44:18.504557 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:44:18.504515    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:44:18.506825 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:44:18.506800    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:44:29.550151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:44:29.550111    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:39.751641 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:44:39.751598    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:48.360592 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:44:48.360541    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:44:56.051289 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:44:56.051254    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"]
Oct 30 22:44:56.100830 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:44:56.100789    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2b53613-xq85k"]
Oct 30 22:44:57.446599 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:44:57.446566    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5428150b-ed70-4c23-bf07-56b07d376863" path="/var/lib/kubelet/pods/5428150b-ed70-4c23-bf07-56b07d376863/volumes"
Oct 30 22:45:08.607782 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:45:08.607690    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-685df8ccfd-7w6pg_f60cdd87-038a-4f42-a56d-c2d3f32911ef/authorino/0.log"
Oct 30 22:45:18.361012 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:45:18.360967    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:45:21.632934 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:45:21.632856    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:21.906491 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:45:21.906460    5586 scope.go:122] "RemoveContainer" containerID="df2f1b6d2848f7ca8e387fb6d3c05877caeaae39fe708eabf07a634d17a1b506"
Oct 30 22:45:21.915760 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:45:21.915738    5586 scope.go:122] "RemoveContainer" containerID="680e7b760bdf56812741bf8a45387a440d13c89de07f2666ed54a7212d66e88b"
Oct 30 22:45:29.577366 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:45:29.577331    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-f74d5f7c9-mzjzm_b2c4b8e3-b6db-4489-9344-6b3fa5bdd778/manager/0.log"
Oct 30 22:45:31.719061 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:45:31.719025    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-mzp2h_bc9eae3c-37d2-4ae7-a9cd-92419715e4eb/server/0.log"
Oct 30 22:45:33.460198 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:45:33.460167    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54959df7f7-w7d92_c9f770df-e0dd-42bd-b5e3-b777bfd20691/maas-api/0.log"
Oct 30 22:45:36.006467 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:45:36.006435    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-685df8ccfd-7w6pg_f60cdd87-038a-4f42-a56d-c2d3f32911ef/authorino/0.log"
Oct 30 22:45:36.803502 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:45:36.803474    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-xx2qj_692e7194-b0df-4f38-8ea9-4fd9abff8e97/manager/0.log"
Oct 30 22:45:37.035051 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:45:37.035020    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-q66hq_54e53152-2818-40cb-a054-c313ce649d24/kuadrant-console-plugin/0.log"
Oct 30 22:45:37.281406 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:45:37.281372    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-gz4nl_1288e96c-85cb-4e3a-a566-ea9253b8307a/manager/0.log"
Oct 30 22:45:40.108297 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:45:40.108260    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:48.361728 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:45:48.361687    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:46:09.729526 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:09.729489    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c46d956fb-t9g2k_344bab0c-773b-45e2-bf3d-aad88f636c34/kube-auth-proxy/0.log"
Oct 30 22:46:09.978467 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:09.978435    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-46ldm_b1dbe21e-18b9-4af9-b2c2-e36c6408543b/istio-proxy/0.log"
Oct 30 22:46:10.245193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:10.245156    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-55hwj_c88a6fd6-2cac-4ad4-b523-905ed7d62bac/istio-proxy/0.log"
Oct 30 22:46:10.799907 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:10.799871    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-f5gbx_17d596a7-1c71-4494-b4f5-d9735c2204b4/istio-proxy/0.log"
Oct 30 22:46:12.108156 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:12.108118    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-wwv4p_7e2fe270-2a33-4648-96a6-b0486a69350d/istio-proxy/0.log"
Oct 30 22:46:12.847341 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:12.847304    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-l4jhz_921e82fc-b274-48e4-ab0e-e8fbbeba9d2b/payload-pre-processing/0.log"
Oct 30 22:46:13.067300 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:13.067269    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-8m8pr_cbef39d1-31a9-4ca6-a5e0-dafaf19ed452/payload-processing/0.log"
Oct 30 22:46:14.654682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:14.654643    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-tksjb_4c8745d6-d399-4ebf-ac9a-856cfae9f873/main/0.log"
Oct 30 22:46:15.371193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:15.371165    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/1.log"
Oct 30 22:46:15.650766 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:15.650678    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw_e7cacb27-5f0a-4d31-b165-6031f3e9226e/main/0.log"
Oct 30 22:46:16.417633 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:16.417599    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:46:17.205256 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:17.205204    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-685df8ccfd-7w6pg_f60cdd87-038a-4f42-a56d-c2d3f32911ef/authorino/0.log"
Oct 30 22:46:18.360705 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:46:18.360670    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:46:21.942105 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:21.942067    5586 scope.go:122] "RemoveContainer" containerID="1e35a4cfc0e4635550bfa1186b55d482fd280853585ed86a7ee02a202f9f9bde"
Oct 30 22:46:36.481935 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:36.481846    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-f74d5f7c9-mzjzm_b2c4b8e3-b6db-4489-9344-6b3fa5bdd778/manager/0.log"
Oct 30 22:46:38.060301 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:38.060267    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-mzp2h_bc9eae3c-37d2-4ae7-a9cd-92419715e4eb/server/0.log"
Oct 30 22:46:39.300894 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:39.300858    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54959df7f7-w7d92_c9f770df-e0dd-42bd-b5e3-b777bfd20691/maas-api/0.log"
Oct 30 22:46:40.440637 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:40.440601    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:41.498296 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:41.498265    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-685df8ccfd-7w6pg_f60cdd87-038a-4f42-a56d-c2d3f32911ef/authorino/0.log"
Oct 30 22:46:42.270864 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:42.270829    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-xx2qj_692e7194-b0df-4f38-8ea9-4fd9abff8e97/manager/0.log"
Oct 30 22:46:42.491469 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:42.491438    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-q66hq_54e53152-2818-40cb-a054-c313ce649d24/kuadrant-console-plugin/0.log"
Oct 30 22:46:42.740723 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:42.740687    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-gz4nl_1288e96c-85cb-4e3a-a566-ea9253b8307a/manager/0.log"
Oct 30 22:46:44.757902 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:44.757868    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c46d956fb-t9g2k_344bab0c-773b-45e2-bf3d-aad88f636c34/kube-auth-proxy/0.log"
Oct 30 22:46:44.985950 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:44.985924    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-46ldm_b1dbe21e-18b9-4af9-b2c2-e36c6408543b/istio-proxy/0.log"
Oct 30 22:46:45.265873 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:45.265843    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-55hwj_c88a6fd6-2cac-4ad4-b523-905ed7d62bac/istio-proxy/0.log"
Oct 30 22:46:45.822202 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:45.822165    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-f5gbx_17d596a7-1c71-4494-b4f5-d9735c2204b4/istio-proxy/0.log"
Oct 30 22:46:47.119520 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:47.119481    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-wwv4p_7e2fe270-2a33-4648-96a6-b0486a69350d/istio-proxy/0.log"
Oct 30 22:46:47.865660 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:47.865625    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-l4jhz_921e82fc-b274-48e4-ab0e-e8fbbeba9d2b/payload-pre-processing/0.log"
Oct 30 22:46:48.076887 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:48.076851    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-8m8pr_cbef39d1-31a9-4ca6-a5e0-dafaf19ed452/payload-processing/0.log"
Oct 30 22:46:48.362049 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:46:48.362014    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:46:49.367957 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:49.367923    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-tksjb_4c8745d6-d399-4ebf-ac9a-856cfae9f873/main/0.log"
Oct 30 22:46:50.065742 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:50.065696    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/1.log"
Oct 30 22:46:50.342930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:46:50.342844    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-j9ncw_e7cacb27-5f0a-4d31-b165-6031f3e9226e/main/0.log"
Oct 30 22:47:18.361391 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:47:18.361347    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:47:30.429080 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:47:30.429050    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-8m8pr_cbef39d1-31a9-4ca6-a5e0-dafaf19ed452/payload-processing/0.log"
Oct 30 22:47:30.657511 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:47:30.657479    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-8m8pr_cbef39d1-31a9-4ca6-a5e0-dafaf19ed452/payload-processing/0.log"
Oct 30 22:47:40.788264 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:47:40.788205    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:48.361458 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:47:48.361417    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:47:52.936542 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:47:52.936503    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:53.149294 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:47:53.149259    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:18.361846 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:48:18.361766    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:48:28.883995 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:48:28.883946    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:41.229976 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:48:41.229933    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:48.361815 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:48:48.361774    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:49:11.414491 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:49:11.414451    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:12.472181 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:49:12.472138    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:18.361636 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:49:18.361599    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:49:18.730554 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:49:18.730518    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:49:18.735350 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:49:18.735324    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:49:18.737219 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:49:18.737195    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-2xrvx_012a98b7-fc3b-43d2-8725-099a8eda00de/main/0.log"
Oct 30 22:49:18.742105 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:49:18.742082    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644svdc4_a1f9a138-94b8-490d-a1eb-32a6ece64758/extract/0.log"
Oct 30 22:49:41.461133 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:49:41.461093    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:48.362151 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:49:48.362106    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:50:18.360658 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:50:18.360619    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:50:19.672968 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:19.672941    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-8m8pr_cbef39d1-31a9-4ca6-a5e0-dafaf19ed452/payload-processing/0.log"
Oct 30 22:50:19.894048 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:19.894019    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-8m8pr_cbef39d1-31a9-4ca6-a5e0-dafaf19ed452/payload-processing/0.log"
Oct 30 22:50:20.271151 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:20.271113    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:20.476975 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:20.476937    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:30.393634 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:30.393603    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ztncc_fde4a00d-016e-4670-a530-f61d1e193a9c/global-pull-secret-syncer/0.log"
Oct 30 22:50:32.422279 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:32.422244    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log"
Oct 30 22:50:33.786191 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:33.786157    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-104.us-west-1.compute.internal_6ea3e08f52bca920602c76ad35bcaf69/haproxy/0.log"
Oct 30 22:50:38.515272 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:38.515210    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-685df8ccfd-7w6pg_f60cdd87-038a-4f42-a56d-c2d3f32911ef/authorino/0.log"
Oct 30 22:50:39.136255 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.136193    5586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"]
Oct 30 22:50:39.140317 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.140292    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.198493 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.198450    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"]
Oct 30 22:50:39.244843 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.244816    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-phvzg\"/\"kube-root-ca.crt\""
Oct 30 22:50:39.245212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.245192    5586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-phvzg\"/\"openshift-service-ca.crt\""
Oct 30 22:50:39.245283 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.245261    5586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-phvzg\"/\"default-dockercfg-t97cc\""
Oct 30 22:50:39.247533 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.247510    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-xx2qj_692e7194-b0df-4f38-8ea9-4fd9abff8e97/manager/0.log"
Oct 30 22:50:39.260678 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.260651    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e43cf33f-83ed-455d-a627-a87663d94912-podres\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.260803 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.260689    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e43cf33f-83ed-455d-a627-a87663d94912-sys\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.260803 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.260707    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e43cf33f-83ed-455d-a627-a87663d94912-lib-modules\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.260879 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.260813    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xck4k\" (UniqueName: \"kubernetes.io/projected/e43cf33f-83ed-455d-a627-a87663d94912-kube-api-access-xck4k\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.260916 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.260875    5586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e43cf33f-83ed-455d-a627-a87663d94912-proc\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.362040 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.362005    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e43cf33f-83ed-455d-a627-a87663d94912-podres\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.362040 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.362051    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e43cf33f-83ed-455d-a627-a87663d94912-sys\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.362312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.362071    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e43cf33f-83ed-455d-a627-a87663d94912-lib-modules\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.362312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.362118    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xck4k\" (UniqueName: \"kubernetes.io/projected/e43cf33f-83ed-455d-a627-a87663d94912-kube-api-access-xck4k\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.362312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.362145    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e43cf33f-83ed-455d-a627-a87663d94912-podres\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.362312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.362171    5586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e43cf33f-83ed-455d-a627-a87663d94912-proc\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.362312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.362144    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e43cf33f-83ed-455d-a627-a87663d94912-sys\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.362312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.362199    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e43cf33f-83ed-455d-a627-a87663d94912-lib-modules\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.362312 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.362225    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e43cf33f-83ed-455d-a627-a87663d94912-proc\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.419225 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.419196    5586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xck4k\" (UniqueName: \"kubernetes.io/projected/e43cf33f-83ed-455d-a627-a87663d94912-kube-api-access-xck4k\") pod \"perf-node-gather-daemonset-ddwh9\" (UID: \"e43cf33f-83ed-455d-a627-a87663d94912\") " pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.453692 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.453662    5586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:39.660476 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.660443    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-q66hq_54e53152-2818-40cb-a054-c313ce649d24/kuadrant-console-plugin/0.log"
Oct 30 22:50:39.828261 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.828221    5586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"]
Oct 30 22:50:39.830906 ip-10-0-130-104 kubenswrapper[5586]: W1030 22:50:39.830877    5586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode43cf33f_83ed_455d_a627_a87663d94912.slice/crio-fc5e42a642be256be2365d6aa934910d1fcd9819378485756a098bbe46ba843e WatchSource:0}: Error finding container fc5e42a642be256be2365d6aa934910d1fcd9819378485756a098bbe46ba843e: Status 404 returned error can't find the container with id fc5e42a642be256be2365d6aa934910d1fcd9819378485756a098bbe46ba843e
Oct 30 22:50:39.949570 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.949530    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9" event={"ID":"e43cf33f-83ed-455d-a627-a87663d94912","Type":"ContainerStarted","Data":"455edab3c5cc7466621122467c33c08199eb848d87832123c3fe2b289775e296"}
Oct 30 22:50:39.949752 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.949595    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9" event={"ID":"e43cf33f-83ed-455d-a627-a87663d94912","Type":"ContainerStarted","Data":"fc5e42a642be256be2365d6aa934910d1fcd9819378485756a098bbe46ba843e"}
Oct 30 22:50:39.949752 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:39.949621    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:40.066361 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:40.066305    5586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9" podStartSLOduration=1.066291422 podStartE2EDuration="1.066291422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:50:40.064342532 +0000 UTC m=+4582.736246470" watchObservedRunningTime="2026-10-30 22:50:40.066291422 +0000 UTC m=+4582.738195386"
Oct 30 22:50:40.123623 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:40.123541    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-gz4nl_1288e96c-85cb-4e3a-a566-ea9253b8307a/manager/0.log"
Oct 30 22:50:41.394457 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:41.394423    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"]
Oct 30 22:50:41.394970 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:41.394647    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" podUID="b1dbe21e-18b9-4af9-b2c2-e36c6408543b" containerName="istio-proxy" containerID="cri-o://6057a9d450a8eeb00e147dcb81ecfe5fc65e6650977bd8ea7c313c31330414b2" gracePeriod=30
Oct 30 22:50:41.446746 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:41.446717    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"]
Oct 30 22:50:41.446930 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:41.446777    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"]
Oct 30 22:50:41.446980 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:41.446918    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx" podUID="17d596a7-1c71-4494-b4f5-d9735c2204b4" containerName="istio-proxy" containerID="cri-o://8ca8d2b203e0f220c49347b6a836a883291dea182656ecafbd70bb611b35c6cd" gracePeriod=30
Oct 30 22:50:41.447017 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:41.446996    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" podUID="7e2fe270-2a33-4648-96a6-b0486a69350d" containerName="istio-proxy" containerID="cri-o://d6e6b22898f93384f1d13c9f5188a4275e40d90c8702583e090766d42e60a6ce" gracePeriod=30
Oct 30 22:50:41.493922 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:41.493883    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"]
Oct 30 22:50:41.494113 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:41.494091    5586 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" podUID="c88a6fd6-2cac-4ad4-b523-905ed7d62bac" containerName="istio-proxy" containerID="cri-o://6571851fb07a096609a8a7ed53f396ee604e0f39e6b1efccdd1b65998c4418af" gracePeriod=30
Oct 30 22:50:41.782884 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:41.782830    5586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9zq7q_3142d576-19ca-48d1-b50f-c22945af6ba3/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:45.965730 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:45.965701    5586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-phvzg/perf-node-gather-daemonset-ddwh9"
Oct 30 22:50:46.711899 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.711200    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:50:46.775912 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.775881    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:50:46.814181 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.814150    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:50:46.822087 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.822066    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:50:46.842370 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.842334    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-kube-api-access-lmggh\" (UniqueName: \"kubernetes.io/projected/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-kube-api-access-lmggh\") pod \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") "
Oct 30 22:50:46.842562 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.842426    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-workload-socket\") pod \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") "
Oct 30 22:50:46.842562 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.842475    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-envoy\") pod \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") "
Oct 30 22:50:46.842562 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.842524    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-data\") pod \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") "
Oct 30 22:50:46.842754 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.842597    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-credential-socket\") pod \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") "
Oct 30 22:50:46.842754 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.842643    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-podinfo\") pod \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") "
Oct 30 22:50:46.842754 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.842723    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-token\" (UniqueName: \"kubernetes.io/projected/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-token\") pod \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") "
Oct 30 22:50:46.842917 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.842757    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istiod-ca-cert\") pod \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") "
Oct 30 22:50:46.842917 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.842787    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-workload-certs\") pod \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\" (UID: \"b1dbe21e-18b9-4af9-b2c2-e36c6408543b\") "
Oct 30 22:50:46.842917 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.842823    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-workload-socket" pod "b1dbe21e-18b9-4af9-b2c2-e36c6408543b" (UID: "b1dbe21e-18b9-4af9-b2c2-e36c6408543b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.842917 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.842862    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-data" pod "b1dbe21e-18b9-4af9-b2c2-e36c6408543b" (UID: "b1dbe21e-18b9-4af9-b2c2-e36c6408543b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.843200 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.843167    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.843200 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.843187    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-workload-certs" pod "b1dbe21e-18b9-4af9-b2c2-e36c6408543b" (UID: "b1dbe21e-18b9-4af9-b2c2-e36c6408543b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.843363 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.843191    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.843363 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.843224    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istiod-ca-cert" pod "b1dbe21e-18b9-4af9-b2c2-e36c6408543b" (UID: "b1dbe21e-18b9-4af9-b2c2-e36c6408543b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:46.843491 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.843440    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-credential-socket" pod "b1dbe21e-18b9-4af9-b2c2-e36c6408543b" (UID: "b1dbe21e-18b9-4af9-b2c2-e36c6408543b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.844816 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.844790    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-kube-api-access-lmggh" pod "b1dbe21e-18b9-4af9-b2c2-e36c6408543b" (UID: "b1dbe21e-18b9-4af9-b2c2-e36c6408543b"). InnerVolumeSpecName "kube-api-access-lmggh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:46.845030 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.845008    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-token" pod "b1dbe21e-18b9-4af9-b2c2-e36c6408543b" (UID: "b1dbe21e-18b9-4af9-b2c2-e36c6408543b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:46.845154 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.845138    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-envoy" pod "b1dbe21e-18b9-4af9-b2c2-e36c6408543b" (UID: "b1dbe21e-18b9-4af9-b2c2-e36c6408543b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.845359 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.845343    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-podinfo" pod "b1dbe21e-18b9-4af9-b2c2-e36c6408543b" (UID: "b1dbe21e-18b9-4af9-b2c2-e36c6408543b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:50:46.872171 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.872133    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-p55vc_c3ed2089-3110-405e-928e-dcdf13e4d4e3/kube-state-metrics/0.log"
Oct 30 22:50:46.944368 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944279    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-credential-socket\") pod \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") "
Oct 30 22:50:46.944368 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944321    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-podinfo\") pod \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") "
Oct 30 22:50:46.944368 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944339    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17d596a7-1c71-4494-b4f5-d9735c2204b4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17d596a7-1c71-4494-b4f5-d9735c2204b4-istiod-ca-cert\") pod \"17d596a7-1c71-4494-b4f5-d9735c2204b4\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") "
Oct 30 22:50:46.944368 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944358    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-data\") pod \"17d596a7-1c71-4494-b4f5-d9735c2204b4\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") "
Oct 30 22:50:46.944682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944546    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-data\") pod \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") "
Oct 30 22:50:46.944682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944600    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-data\") pod \"7e2fe270-2a33-4648-96a6-b0486a69350d\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") "
Oct 30 22:50:46.944682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944611    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-data" pod "17d596a7-1c71-4494-b4f5-d9735c2204b4" (UID: "17d596a7-1c71-4494-b4f5-d9735c2204b4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.944682 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944637    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-kube-api-access-hwnv9\" (UniqueName: \"kubernetes.io/projected/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-kube-api-access-hwnv9\") pod \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") "
Oct 30 22:50:46.944910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944723    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-token\" (UniqueName: \"kubernetes.io/projected/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-token\") pod \"17d596a7-1c71-4494-b4f5-d9735c2204b4\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") "
Oct 30 22:50:46.944910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944646    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-credential-socket" pod "c88a6fd6-2cac-4ad4-b523-905ed7d62bac" (UID: "c88a6fd6-2cac-4ad4-b523-905ed7d62bac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.944910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944766    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e2fe270-2a33-4648-96a6-b0486a69350d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2fe270-2a33-4648-96a6-b0486a69350d-istiod-ca-cert\") pod \"7e2fe270-2a33-4648-96a6-b0486a69350d\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") "
Oct 30 22:50:46.944910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944798    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-workload-certs\") pod \"17d596a7-1c71-4494-b4f5-d9735c2204b4\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") "
Oct 30 22:50:46.944910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944833    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istiod-ca-cert\") pod \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") "
Oct 30 22:50:46.944910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944834    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-data" pod "c88a6fd6-2cac-4ad4-b523-905ed7d62bac" (UID: "c88a6fd6-2cac-4ad4-b523-905ed7d62bac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.944910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944839    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-data" pod "7e2fe270-2a33-4648-96a6-b0486a69350d" (UID: "7e2fe270-2a33-4648-96a6-b0486a69350d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.944910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944866    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-credential-socket\") pod \"17d596a7-1c71-4494-b4f5-d9735c2204b4\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") "
Oct 30 22:50:46.944910 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944906    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-envoy\") pod \"7e2fe270-2a33-4648-96a6-b0486a69350d\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") "
Oct 30 22:50:46.945435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944937    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-workload-socket\") pod \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") "
Oct 30 22:50:46.945435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944968    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-workload-certs\") pod \"7e2fe270-2a33-4648-96a6-b0486a69350d\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") "
Oct 30 22:50:46.945435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944995    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-credential-socket\") pod \"7e2fe270-2a33-4648-96a6-b0486a69350d\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") "
Oct 30 22:50:46.945435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945034    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17d596a7-1c71-4494-b4f5-d9735c2204b4-kube-api-access-bgmg2\" (UniqueName: \"kubernetes.io/projected/17d596a7-1c71-4494-b4f5-d9735c2204b4-kube-api-access-bgmg2\") pod \"17d596a7-1c71-4494-b4f5-d9735c2204b4\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") "
Oct 30 22:50:46.945435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945063    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-workload-certs\") pod \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") "
Oct 30 22:50:46.945435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.944863    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17d596a7-1c71-4494-b4f5-d9735c2204b4-istiod-ca-cert" pod "17d596a7-1c71-4494-b4f5-d9735c2204b4" (UID: "17d596a7-1c71-4494-b4f5-d9735c2204b4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:46.945435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945055    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-workload-certs" pod "17d596a7-1c71-4494-b4f5-d9735c2204b4" (UID: "17d596a7-1c71-4494-b4f5-d9735c2204b4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.945435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945175    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-token\" (UniqueName: \"kubernetes.io/projected/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-token\") pod \"7e2fe270-2a33-4648-96a6-b0486a69350d\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") "
Oct 30 22:50:46.945435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945286    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-workload-socket\") pod \"17d596a7-1c71-4494-b4f5-d9735c2204b4\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") "
Oct 30 22:50:46.945435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945325    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-token\" (UniqueName: \"kubernetes.io/projected/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-token\") pod \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") "
Oct 30 22:50:46.945435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945370    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-podinfo\") pod \"17d596a7-1c71-4494-b4f5-d9735c2204b4\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") "
Oct 30 22:50:46.945435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945408    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-envoy\") pod \"17d596a7-1c71-4494-b4f5-d9735c2204b4\" (UID: \"17d596a7-1c71-4494-b4f5-d9735c2204b4\") "
Oct 30 22:50:46.945435 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945441    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-envoy\") pod \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\" (UID: \"c88a6fd6-2cac-4ad4-b523-905ed7d62bac\") "
Oct 30 22:50:46.946148 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945483    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-podinfo\") pod \"7e2fe270-2a33-4648-96a6-b0486a69350d\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") "
Oct 30 22:50:46.946148 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945528    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e2fe270-2a33-4648-96a6-b0486a69350d-kube-api-access-mc6sr\" (UniqueName: \"kubernetes.io/projected/7e2fe270-2a33-4648-96a6-b0486a69350d-kube-api-access-mc6sr\") pod \"7e2fe270-2a33-4648-96a6-b0486a69350d\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") "
Oct 30 22:50:46.946148 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945172    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e2fe270-2a33-4648-96a6-b0486a69350d-istiod-ca-cert" pod "7e2fe270-2a33-4648-96a6-b0486a69350d" (UID: "7e2fe270-2a33-4648-96a6-b0486a69350d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:46.946148 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945562    5586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-workload-socket\") pod \"7e2fe270-2a33-4648-96a6-b0486a69350d\" (UID: \"7e2fe270-2a33-4648-96a6-b0486a69350d\") "
Oct 30 22:50:46.946148 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945175    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istiod-ca-cert" pod "c88a6fd6-2cac-4ad4-b523-905ed7d62bac" (UID: "c88a6fd6-2cac-4ad4-b523-905ed7d62bac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:50:46.946148 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945300    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-workload-socket" pod "c88a6fd6-2cac-4ad4-b523-905ed7d62bac" (UID: "c88a6fd6-2cac-4ad4-b523-905ed7d62bac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.946148 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945364    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-workload-certs" pod "7e2fe270-2a33-4648-96a6-b0486a69350d" (UID: "7e2fe270-2a33-4648-96a6-b0486a69350d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.946148 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945531    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-credential-socket" pod "7e2fe270-2a33-4648-96a6-b0486a69350d" (UID: "7e2fe270-2a33-4648-96a6-b0486a69350d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.946148 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945543    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-workload-certs" pod "c88a6fd6-2cac-4ad4-b523-905ed7d62bac" (UID: "c88a6fd6-2cac-4ad4-b523-905ed7d62bac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.946148 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945632    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-workload-socket" pod "17d596a7-1c71-4494-b4f5-d9735c2204b4" (UID: "17d596a7-1c71-4494-b4f5-d9735c2204b4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.946148 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.945097    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-credential-socket" pod "17d596a7-1c71-4494-b4f5-d9735c2204b4" (UID: "17d596a7-1c71-4494-b4f5-d9735c2204b4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946523    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17d596a7-1c71-4494-b4f5-d9735c2204b4-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946544    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946560    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946573    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946587    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946600    5586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-data\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946615    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2fe270-2a33-4648-96a6-b0486a69350d-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946629    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946645    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946659    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946672    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946686    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946708    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946722    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946737    5586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istiod-ca-cert\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946750    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946763    5586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-workload-certs\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946777    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmggh\" (UniqueName: \"kubernetes.io/projected/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-kube-api-access-lmggh\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.946791 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946791    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.947824 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946805    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1dbe21e-18b9-4af9-b2c2-e36c6408543b-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.947824 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.946818    5586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-credential-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:46.947824 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.947128    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-workload-socket" pod "7e2fe270-2a33-4648-96a6-b0486a69350d" (UID: "7e2fe270-2a33-4648-96a6-b0486a69350d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.948016 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.947984    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-kube-api-access-hwnv9" pod "c88a6fd6-2cac-4ad4-b523-905ed7d62bac" (UID: "c88a6fd6-2cac-4ad4-b523-905ed7d62bac"). InnerVolumeSpecName "kube-api-access-hwnv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:46.948076 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.948041    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-envoy" pod "7e2fe270-2a33-4648-96a6-b0486a69350d" (UID: "7e2fe270-2a33-4648-96a6-b0486a69350d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.948133 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.948088    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17d596a7-1c71-4494-b4f5-d9735c2204b4-kube-api-access-bgmg2" pod "17d596a7-1c71-4494-b4f5-d9735c2204b4" (UID: "17d596a7-1c71-4494-b4f5-d9735c2204b4"). InnerVolumeSpecName "kube-api-access-bgmg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:46.948912 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.948871    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-token" pod "7e2fe270-2a33-4648-96a6-b0486a69350d" (UID: "7e2fe270-2a33-4648-96a6-b0486a69350d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:46.949481 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.949424    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-envoy" pod "c88a6fd6-2cac-4ad4-b523-905ed7d62bac" (UID: "c88a6fd6-2cac-4ad4-b523-905ed7d62bac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.949731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.949485    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e2fe270-2a33-4648-96a6-b0486a69350d-kube-api-access-mc6sr" pod "7e2fe270-2a33-4648-96a6-b0486a69350d" (UID: "7e2fe270-2a33-4648-96a6-b0486a69350d"). InnerVolumeSpecName "kube-api-access-mc6sr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:46.950372 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.950346    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-token" pod "17d596a7-1c71-4494-b4f5-d9735c2204b4" (UID: "17d596a7-1c71-4494-b4f5-d9735c2204b4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:46.950662 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.950631    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-token" pod "c88a6fd6-2cac-4ad4-b523-905ed7d62bac" (UID: "c88a6fd6-2cac-4ad4-b523-905ed7d62bac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:46.950797 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.950727    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-envoy" pod "17d596a7-1c71-4494-b4f5-d9735c2204b4" (UID: "17d596a7-1c71-4494-b4f5-d9735c2204b4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:50:46.950797 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.950756    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-podinfo" pod "c88a6fd6-2cac-4ad4-b523-905ed7d62bac" (UID: "c88a6fd6-2cac-4ad4-b523-905ed7d62bac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:50:46.950897 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.950837    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-podinfo" pod "7e2fe270-2a33-4648-96a6-b0486a69350d" (UID: "7e2fe270-2a33-4648-96a6-b0486a69350d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:50:46.951037 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.951018    5586 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-podinfo" pod "17d596a7-1c71-4494-b4f5-d9735c2204b4" (UID: "17d596a7-1c71-4494-b4f5-d9735c2204b4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:50:46.990189 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.989983    5586 generic.go:356] "Generic (PLEG): container finished" podID="7e2fe270-2a33-4648-96a6-b0486a69350d" containerID="d6e6b22898f93384f1d13c9f5188a4275e40d90c8702583e090766d42e60a6ce" exitCode=0
Oct 30 22:50:46.990189 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.990058    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"
Oct 30 22:50:46.990189 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.990078    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" event={"ID":"7e2fe270-2a33-4648-96a6-b0486a69350d","Type":"ContainerDied","Data":"d6e6b22898f93384f1d13c9f5188a4275e40d90c8702583e090766d42e60a6ce"}
Oct 30 22:50:46.990189 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.990144    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p" event={"ID":"7e2fe270-2a33-4648-96a6-b0486a69350d","Type":"ContainerDied","Data":"a7dbadaa2d2b4ce5b46dac76abb28bf8e9c6ee4f67af1ecf667dd2350f137875"}
Oct 30 22:50:46.990189 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.990169    5586 scope.go:122] "RemoveContainer" containerID="d6e6b22898f93384f1d13c9f5188a4275e40d90c8702583e090766d42e60a6ce"
Oct 30 22:50:46.995604 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.995579    5586 generic.go:356] "Generic (PLEG): container finished" podID="c88a6fd6-2cac-4ad4-b523-905ed7d62bac" containerID="6571851fb07a096609a8a7ed53f396ee604e0f39e6b1efccdd1b65998c4418af" exitCode=0
Oct 30 22:50:46.995751 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.995634    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"
Oct 30 22:50:46.995751 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.995664    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" event={"ID":"c88a6fd6-2cac-4ad4-b523-905ed7d62bac","Type":"ContainerDied","Data":"6571851fb07a096609a8a7ed53f396ee604e0f39e6b1efccdd1b65998c4418af"}
Oct 30 22:50:46.995751 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.995712    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj" event={"ID":"c88a6fd6-2cac-4ad4-b523-905ed7d62bac","Type":"ContainerDied","Data":"e46a24f623f8e46b43ed257c0f5faa7bd6a50609129971f9c9731248144f32ca"}
Oct 30 22:50:46.997314 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.997289    5586 generic.go:356] "Generic (PLEG): container finished" podID="17d596a7-1c71-4494-b4f5-d9735c2204b4" containerID="8ca8d2b203e0f220c49347b6a836a883291dea182656ecafbd70bb611b35c6cd" exitCode=0
Oct 30 22:50:46.997427 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.997347    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"
Oct 30 22:50:46.997427 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.997355    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx" event={"ID":"17d596a7-1c71-4494-b4f5-d9735c2204b4","Type":"ContainerDied","Data":"8ca8d2b203e0f220c49347b6a836a883291dea182656ecafbd70bb611b35c6cd"}
Oct 30 22:50:46.997427 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.997401    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx" event={"ID":"17d596a7-1c71-4494-b4f5-d9735c2204b4","Type":"ContainerDied","Data":"b1351638eb1f3a5e103ab5f47b33a15bc116e338f6fcf0025e8dec93225eb5a2"}
Oct 30 22:50:46.999102 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.999079    5586 generic.go:356] "Generic (PLEG): container finished" podID="b1dbe21e-18b9-4af9-b2c2-e36c6408543b" containerID="6057a9d450a8eeb00e147dcb81ecfe5fc65e6650977bd8ea7c313c31330414b2" exitCode=0
Oct 30 22:50:46.999206 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.999133    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" event={"ID":"b1dbe21e-18b9-4af9-b2c2-e36c6408543b","Type":"ContainerDied","Data":"6057a9d450a8eeb00e147dcb81ecfe5fc65e6650977bd8ea7c313c31330414b2"}
Oct 30 22:50:46.999206 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.999170    5586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm" event={"ID":"b1dbe21e-18b9-4af9-b2c2-e36c6408543b","Type":"ContainerDied","Data":"9f1288a7991605132546c81b27c9030de4e773473148b8ffbd6174a0952f6249"}
Oct 30 22:50:46.999399 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:46.999250    5586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"
Oct 30 22:50:47.011987 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.011917    5586 scope.go:122] "RemoveContainer" containerID="d6e6b22898f93384f1d13c9f5188a4275e40d90c8702583e090766d42e60a6ce"
Oct 30 22:50:47.012331 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:50:47.012304    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6e6b22898f93384f1d13c9f5188a4275e40d90c8702583e090766d42e60a6ce\": container with ID starting with d6e6b22898f93384f1d13c9f5188a4275e40d90c8702583e090766d42e60a6ce not found: ID does not exist" containerID="d6e6b22898f93384f1d13c9f5188a4275e40d90c8702583e090766d42e60a6ce"
Oct 30 22:50:47.012427 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.012344    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6e6b22898f93384f1d13c9f5188a4275e40d90c8702583e090766d42e60a6ce"} err="failed to get container status \"d6e6b22898f93384f1d13c9f5188a4275e40d90c8702583e090766d42e60a6ce\": rpc error: code = NotFound desc = could not find container \"d6e6b22898f93384f1d13c9f5188a4275e40d90c8702583e090766d42e60a6ce\": container with ID starting with d6e6b22898f93384f1d13c9f5188a4275e40d90c8702583e090766d42e60a6ce not found: ID does not exist"
Oct 30 22:50:47.012427 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.012369    5586 scope.go:122] "RemoveContainer" containerID="6571851fb07a096609a8a7ed53f396ee604e0f39e6b1efccdd1b65998c4418af"
Oct 30 22:50:47.023939 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.023916    5586 scope.go:122] "RemoveContainer" containerID="6571851fb07a096609a8a7ed53f396ee604e0f39e6b1efccdd1b65998c4418af"
Oct 30 22:50:47.024263 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:50:47.024243    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6571851fb07a096609a8a7ed53f396ee604e0f39e6b1efccdd1b65998c4418af\": container with ID starting with 6571851fb07a096609a8a7ed53f396ee604e0f39e6b1efccdd1b65998c4418af not found: ID does not exist" containerID="6571851fb07a096609a8a7ed53f396ee604e0f39e6b1efccdd1b65998c4418af"
Oct 30 22:50:47.024313 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.024272    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6571851fb07a096609a8a7ed53f396ee604e0f39e6b1efccdd1b65998c4418af"} err="failed to get container status \"6571851fb07a096609a8a7ed53f396ee604e0f39e6b1efccdd1b65998c4418af\": rpc error: code = NotFound desc = could not find container \"6571851fb07a096609a8a7ed53f396ee604e0f39e6b1efccdd1b65998c4418af\": container with ID starting with 6571851fb07a096609a8a7ed53f396ee604e0f39e6b1efccdd1b65998c4418af not found: ID does not exist"
Oct 30 22:50:47.024313 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.024289    5586 scope.go:122] "RemoveContainer" containerID="8ca8d2b203e0f220c49347b6a836a883291dea182656ecafbd70bb611b35c6cd"
Oct 30 22:50:47.033795 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.033775    5586 scope.go:122] "RemoveContainer" containerID="8ca8d2b203e0f220c49347b6a836a883291dea182656ecafbd70bb611b35c6cd"
Oct 30 22:50:47.034100 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:50:47.034080    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ca8d2b203e0f220c49347b6a836a883291dea182656ecafbd70bb611b35c6cd\": container with ID starting with 8ca8d2b203e0f220c49347b6a836a883291dea182656ecafbd70bb611b35c6cd not found: ID does not exist" containerID="8ca8d2b203e0f220c49347b6a836a883291dea182656ecafbd70bb611b35c6cd"
Oct 30 22:50:47.034177 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.034111    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ca8d2b203e0f220c49347b6a836a883291dea182656ecafbd70bb611b35c6cd"} err="failed to get container status \"8ca8d2b203e0f220c49347b6a836a883291dea182656ecafbd70bb611b35c6cd\": rpc error: code = NotFound desc = could not find container \"8ca8d2b203e0f220c49347b6a836a883291dea182656ecafbd70bb611b35c6cd\": container with ID starting with 8ca8d2b203e0f220c49347b6a836a883291dea182656ecafbd70bb611b35c6cd not found: ID does not exist"
Oct 30 22:50:47.034177 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.034137    5586 scope.go:122] "RemoveContainer" containerID="6057a9d450a8eeb00e147dcb81ecfe5fc65e6650977bd8ea7c313c31330414b2"
Oct 30 22:50:47.042842 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.042819    5586 scope.go:122] "RemoveContainer" containerID="6057a9d450a8eeb00e147dcb81ecfe5fc65e6650977bd8ea7c313c31330414b2"
Oct 30 22:50:47.043126 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:50:47.043106    5586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6057a9d450a8eeb00e147dcb81ecfe5fc65e6650977bd8ea7c313c31330414b2\": container with ID starting with 6057a9d450a8eeb00e147dcb81ecfe5fc65e6650977bd8ea7c313c31330414b2 not found: ID does not exist" containerID="6057a9d450a8eeb00e147dcb81ecfe5fc65e6650977bd8ea7c313c31330414b2"
Oct 30 22:50:47.043181 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.043133    5586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6057a9d450a8eeb00e147dcb81ecfe5fc65e6650977bd8ea7c313c31330414b2"} err="failed to get container status \"6057a9d450a8eeb00e147dcb81ecfe5fc65e6650977bd8ea7c313c31330414b2\": rpc error: code = NotFound desc = could not find container \"6057a9d450a8eeb00e147dcb81ecfe5fc65e6650977bd8ea7c313c31330414b2\": container with ID starting with 6057a9d450a8eeb00e147dcb81ecfe5fc65e6650977bd8ea7c313c31330414b2 not found: ID does not exist"
Oct 30 22:50:47.047628 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.047605    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwnv9\" (UniqueName: \"kubernetes.io/projected/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-kube-api-access-hwnv9\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:47.047731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.047631    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:47.047731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.047645    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:47.047731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.047657    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgmg2\" (UniqueName: \"kubernetes.io/projected/17d596a7-1c71-4494-b4f5-d9735c2204b4-kube-api-access-bgmg2\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:47.047731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.047674    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:47.047731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.047689    5586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-token\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:47.047731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.047702    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:47.047731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.047715    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17d596a7-1c71-4494-b4f5-d9735c2204b4-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:47.047731 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.047728    5586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-envoy\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:47.048023 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.047741    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2fe270-2a33-4648-96a6-b0486a69350d-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:47.048023 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.047758    5586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mc6sr\" (UniqueName: \"kubernetes.io/projected/7e2fe270-2a33-4648-96a6-b0486a69350d-kube-api-access-mc6sr\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:47.048023 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.047776    5586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2fe270-2a33-4648-96a6-b0486a69350d-workload-socket\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:47.048023 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.047789    5586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c88a6fd6-2cac-4ad4-b523-905ed7d62bac-istio-podinfo\") on node \"ip-10-0-130-104.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:50:47.278189 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.278101    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-p55vc_c3ed2089-3110-405e-928e-dcdf13e4d4e3/kube-rbac-proxy-main/0.log"
Oct 30 22:50:47.359439 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.359402    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"]
Oct 30 22:50:47.368223 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.368189    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-46ldm"]
Oct 30 22:50:47.447841 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.447808    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1dbe21e-18b9-4af9-b2c2-e36c6408543b" path="/var/lib/kubelet/pods/b1dbe21e-18b9-4af9-b2c2-e36c6408543b/volumes"
Oct 30 22:50:47.481456 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.481424    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"]
Oct 30 22:50:47.487835 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.487806    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-55hwj"]
Oct 30 22:50:47.597086 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.596997    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"]
Oct 30 22:50:47.599024 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.599001    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-wwv4p"]
Oct 30 22:50:47.693382 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.693351    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-p55vc_c3ed2089-3110-405e-928e-dcdf13e4d4e3/kube-rbac-proxy-self/0.log"
Oct 30 22:50:47.710350 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.710314    5586 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"]
Oct 30 22:50:47.715543 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:47.715514    5586 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-f5gbx"]
Oct 30 22:50:48.364542 ip-10-0-130-104 kubenswrapper[5586]: E1030 22:50:48.364435    5586 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-csjtg" containerName="extractor"
Oct 30 22:50:48.525859 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:48.525826    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-btfmp_7f0cf155-c021-465e-a08b-eb2e2aa0935e/monitoring-plugin/0.log"
Oct 30 22:50:49.446212 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:49.446179    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17d596a7-1c71-4494-b4f5-d9735c2204b4" path="/var/lib/kubelet/pods/17d596a7-1c71-4494-b4f5-d9735c2204b4/volumes"
Oct 30 22:50:49.446640 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:49.446625    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e2fe270-2a33-4648-96a6-b0486a69350d" path="/var/lib/kubelet/pods/7e2fe270-2a33-4648-96a6-b0486a69350d/volumes"
Oct 30 22:50:49.447028 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:49.447016    5586 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c88a6fd6-2cac-4ad4-b523-905ed7d62bac" path="/var/lib/kubelet/pods/c88a6fd6-2cac-4ad4-b523-905ed7d62bac/volumes"
Oct 30 22:50:51.470580 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:51.470549    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zts42_42caab53-09ce-4c8f-b7ab-5d86ae4b64ba/node-exporter/0.log"
Oct 30 22:50:51.901988 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:51.901896    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zts42_42caab53-09ce-4c8f-b7ab-5d86ae4b64ba/kube-rbac-proxy/0.log"
Oct 30 22:50:52.315854 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:52.315823    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zts42_42caab53-09ce-4c8f-b7ab-5d86ae4b64ba/init-textfile/0.log"
Oct 30 22:50:52.732960 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:52.732933    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-2rvck_b40cbded-e2dc-420d-b454-eec7a903571e/kube-rbac-proxy-main/0.log"
Oct 30 22:50:53.146275 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:53.146179    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-2rvck_b40cbded-e2dc-420d-b454-eec7a903571e/kube-rbac-proxy-self/0.log"
Oct 30 22:50:53.559610 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:53.559573    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-2rvck_b40cbded-e2dc-420d-b454-eec7a903571e/openshift-state-metrics/0.log"
Oct 30 22:50:53.984150 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:53.984119    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_76afe9bb-fda3-4e8d-a83e-a06457b7071d/prometheus/0.log"
Oct 30 22:50:54.400900 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:54.400825    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_76afe9bb-fda3-4e8d-a83e-a06457b7071d/config-reloader/0.log"
Oct 30 22:50:54.816293 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:54.816260    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_76afe9bb-fda3-4e8d-a83e-a06457b7071d/thanos-sidecar/0.log"
Oct 30 22:50:55.229527 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:55.229495    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_76afe9bb-fda3-4e8d-a83e-a06457b7071d/kube-rbac-proxy-web/0.log"
Oct 30 22:50:55.642193 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:55.642101    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_76afe9bb-fda3-4e8d-a83e-a06457b7071d/kube-rbac-proxy/0.log"
Oct 30 22:50:56.056415 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:56.056382    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_76afe9bb-fda3-4e8d-a83e-a06457b7071d/kube-rbac-proxy-thanos/0.log"
Oct 30 22:50:56.470639 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:56.470588    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_76afe9bb-fda3-4e8d-a83e-a06457b7071d/init-config-reloader/0.log"
Oct 30 22:50:59.400307 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:59.400189    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5488c7d6f4-h5p6s_17bdd0ae-091c-4442-a4d4-4414e8feb7b2/thanos-query/0.log"
Oct 30 22:50:59.812384 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:50:59.812334    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5488c7d6f4-h5p6s_17bdd0ae-091c-4442-a4d4-4414e8feb7b2/kube-rbac-proxy-web/0.log"
Oct 30 22:51:00.223421 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:51:00.223388    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5488c7d6f4-h5p6s_17bdd0ae-091c-4442-a4d4-4414e8feb7b2/kube-rbac-proxy/0.log"
Oct 30 22:51:00.647011 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:51:00.646935    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5488c7d6f4-h5p6s_17bdd0ae-091c-4442-a4d4-4414e8feb7b2/prom-label-proxy/0.log"
Oct 30 22:51:01.059436 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:51:01.059410    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5488c7d6f4-h5p6s_17bdd0ae-091c-4442-a4d4-4414e8feb7b2/kube-rbac-proxy-rules/0.log"
Oct 30 22:51:01.515530 ip-10-0-130-104 kubenswrapper[5586]: I1030 22:51:01.515494    5586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5488c7d6f4-h5p6s_17bdd0ae-091c-4442-a4d4-4414e8feb7b2/kube-rbac-proxy-metrics/0.log"